Method and apparatus for encoding video based on scanning order of hierarchical data units, and method and apparatus for decoding video based on scanning order of hierarchical data units

ABSTRACT

A video decoding method and apparatus and a video encoding method and apparatus based on a scanning order of hierarchical data units are provided. The decoding method includes: receiving and parsing a bitstream of an encoded video; extracting from the bitstream information about a size of a maximum coding unit for decoding a picture of the encoded video, and encoding information about a coded depth and an encoding mode for coding units of the picture, wherein the size of the maximum coding unit is a maximum size of a coding unit which is a data unit for decoding the picture; and determining a hierarchical structure of the maximum coding unit and the coding units into which the picture is divided according to depths, and decoding the picture based on the coding units, by using the information about the size of the maximum coding unit and the encoding information about the coded depth and the encoded mode.

CROSS-REFERENCE TO RELATED PATENT APPLICATION

This is a continuation application of U.S. patent application Ser. No.15/681,938, filed on Aug. 21, 2017, which is a continuation applicationof U.S. patent application Ser. No. 15/140,731, filed on Apr. 28, 2016,issued as U.S. Pat. No. 9,743,109, which is a continuation applicationof U.S. patent application Ser. No. 14/603,178, filed on Jan. 22, 2015,issued as U.S. Pat. No. 9,351,015, which is a continuation applicationof U.S. patent application Ser. No. 14/305,211, filed on Jun. 16, 2014,issued as U.S. Pat. No. 9,225,997, which is a continuation applicationof U.S. patent application Ser. No. 13/019,343, filed on Feb. 2, 2011,issued as U.S. Pat. No. 8,792,547, which claims priority from KoreanPatent Application No. 10-2010-0009642, filed on Feb. 2, 2010, in theKorean Intellectual Property Office, the disclosures of which areincorporated herein by reference in their entireties.

BACKGROUND 1. Field

Apparatuses and methods consistent with exemplary embodiments relate toencoding and decoding a video.

2. Description of the Related Art

As hardware for reproducing and storing high resolution or high qualityvideo content is being developed and supplied, a need for a video codecfor effectively encoding or decoding the high resolution or high qualityvideo content is increasing. In a related art video codec, a video isencoded according to a limited encoding method based on a macroblockhaving a predetermined size.

SUMMARY

Aspects of exemplary embodiments provide video encoding and decodingbased on a scanning order that depends on absolute locations andrelative locations of coding units and data units that constitute ahierarchical structure.

According to an aspect of an exemplary embodiment, there is provided avideo decoding method based on a scanning order of hierarchical dataunits, the method including: receiving and parsing a bitstream of anencoded video; extracting, from the bitstream, information about a sizeof a maximum coding unit for decoding a picture of the encoded video,and encoding information about a coded depth and an encoding mode forcoding units of the picture, wherein the size of the maximum coding unitis a maximum size of a coding unit which is a data unit for decoding thepicture; and determining a hierarchical structure of the maximum codingunit and the coding units into which the picture is divided according todepths, and decoding the picture based on the coding units, by using theinformation about the size of the maximum coding unit and the encodinginformation about the coded depth and the encoded mode, wherein a datascanning order of an upper data unit and a lower data unit included inthe hierarchical structure of the maximum coding unit and the codingunits is determined based on an absolute location or a relative locationof the lower data unit in comparison to a location of the upper dataunit.

Based on the hierarchical structure of the maximum coding unit and thecoding units, the lower data unit of the maximum coding unit may includeat least one of deeper coding units according to the depths, a codingunit of the coded depth, a prediction unit for prediction-decoding thecoding unit, a partition into which the prediction unit is split, atransformation unit for inversely transforming the coding unit, and aminimum unit, and the upper data unit may include at least one lowerdata unit in an order of the maximum coding unit, the deeper codingunits according to the depths, the coding unit of the coded depth, theprediction unit, the partition, and the minimum unit.

The decoding may include determining a relative location of the lowercoding unit with respect to the maximum coding unit or an absolutelocation of the lower coding unit in the picture by using a relativelocation of the upper coding unit with respect to the maximum codingunit or an absolute location of the upper coding unit in the picture.

According to an aspect of another exemplary embodiment, there isprovided a video encoding method based on a scanning order ofhierarchical data units, the method including: splitting a picture intoa maximum coding unit which has a maximum size and is a data unit inwhich the picture is encoded; determining the maximum coding unit andthe coding units having a hierarchical structure and determining a codeddepth and an encoding mode of the coding units, by encoding the picturebased on the maximum coding unit and the coding units, into which thepicture is split according to depths; and outputting encodinginformation about the coded depth and the encoding mode and sizeinformation of the maximum coding unit, wherein a data scanning order ofdata units belonging to the hierarchical structure of the maximum codingunit and the coding units is determined based on an absolute position ora relative position of a lower data unit as compared to an upper dataunit.

According to an aspect of another exemplary embodiment, there isprovided a video decoding apparatus based on a scanning order ofhierarchical data units, the apparatus including: a receiver whichreceives and parses a bitstream for an encoded video; an extractor whichextracts, from the bitstream, information about a size of a maximumcoding unit for decoding a picture of the encoded video, and encodinginformation about a coded depth and an encoding mode for coding units ofthe picture, wherein the size of the maximum coding unit is a maximumsize of a coding unit which is a data unit for decoding the picture; anda decoder which determines a hierarchical structure of the maximumcoding unit and the coding units into which the picture is dividedaccording to depths, and decodes the picture based on the coding units,by using the information about the size of the maximum coding unit andthe encoding information about the coded depth and the encoded mode,wherein a data scanning order of an upper data unit and a lower dataunit included in the hierarchical structure of the maximum coding unitand the coding units is determined based on an absolute location or arelative location of the lower data unit in comparison to a location ofthe upper data unit.

According to an aspect of another exemplary embodiment, there isprovided a video encoding apparatus based on a scanning order ofhierarchical data units, the apparatus including: a maximum coding unitsplitter which splits a picture into a maximum coding unit which has amaximum size and is a data unit in which the picture is encoded; acoding unit determiner which determines the maximum coding unit and thecoding units having a hierarchical structure and determines a codeddepth and an encoding mode of the coding units, by encoding the picturebased on the maximum coding unit and the coding units, into which thepicture is split according to depths; and an information output unitwhich outputs encoding information about the coded depth and theencoding mode and size information of the maximum coding unit, wherein adata scanning order of data units belonging to the hierarchicalstructure of the maximum coding unit and the coding units is determinedbased on an absolute position or a relative position of a lower dataunit as compared to an upper data unit.

According to an aspect of another exemplary embodiment, there isprovided a computer-readable recording medium having recorded thereon aprogram for executing the video encoding method.

According to an aspect of another exemplary embodiment, there isprovided a computer-readable recording medium having recorded thereon aprogram for executing the video decoding method.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other features will become more apparent by describing indetail exemplary embodiments with reference to the attached drawings inwhich:

FIG. 1 is a block diagram of an apparatus for encoding a video,according to an exemplary embodiment;

FIG. 2 is a block diagram of an apparatus for decoding a video,according to an exemplary embodiment;

FIG. 3 is a diagram for describing a concept of coding units accordingto an exemplary embodiment;

FIG. 4 is a block diagram of an image encoder based on coding unitsaccording to an exemplary embodiment;

FIG. 5 is a block diagram of an image decoder based on coding unitsaccording to an exemplary embodiment;

FIG. 6 is a diagram illustrating deeper coding units according to depthsand prediction units according to an exemplary embodiment;

FIG. 7 is a diagram for describing a relationship between a coding unitand transformation units, according to an exemplary embodiment;

FIG. 8 is a diagram for describing encoding information of coding unitscorresponding to a coded depth, according to an exemplary embodiment;

FIG. 9 is a diagram of deeper coding units according to depths,according to an exemplary embodiment;

FIGS. 10, 11 and 12 are diagrams for describing a relationship betweencoding units, prediction units, and frequency transformation units,according to an exemplary embodiment;

FIG. 13 illustrates a diagram for describing a relationship between acoding unit, a prediction unit or a partition, and a transformationunit, according to encoding mode information of Table 1 according to anexemplary embodiment;

FIG. 14 is a flowchart illustrating a method of encoding a video,according to an exemplary embodiment;

FIG. 15 is a flowchart illustrating a method of decoding a video,according to an exemplary embodiment;

FIG. 16 is a block diagram of an apparatus for encoding a video based ona scanning order of hierarchical data units, according to an exemplaryembodiment;

FIG. 17 is a block diagram of an apparatus for decoding a video based ona scanning order of hierarchical data units, according to an exemplaryembodiment;

FIG. 18 illustrates a relationship between locations and scan indices ofa slice, a coding unit, a prediction unit, a partition, and atransformation unit, according to an exemplary embodiment;

FIG. 19 illustrates indices based on a zigzag scanning order ofhierarchical coding units, according to an exemplary embodiment;

FIG. 20 illustrates relative locations of a maximum coding unit,hierarchical data units, and a minimum unit, according to an exemplaryembodiment;

FIG. 21 illustrates adjacent prediction units for arbitrary directiveintra prediction according to an exemplary embodiment;

FIG. 22 illustrates locations of minimum units in a 2N×2N sized maximumcoding unit and a prediction unit of a partition type 2N×2N, accordingto an exemplary embodiment;

FIG. 23 illustrates locations of minimum units in the 2N×2N sizedmaximum coding unit and prediction units of a partition type 2N×Naccording to an exemplary embodiment;

FIG. 24 illustrates locations of minimum units in the 2N×2N sizedmaximum coding unit and prediction units of a partition type N×2Naccording to an exemplary embodiment;

FIG. 25 illustrates locations of minimum units in the 2N×2N sizedmaximum coding unit and prediction units of a partition type N×Naccording to an exemplary embodiment;

FIG. 26 illustrates locations of minimum units in the 2N×2N sizedmaximum coding unit and prediction units of a partition type 2N×nUaccording to an exemplary embodiment;

FIG. 27 illustrates locations of minimum units in the 2N×2N sizedmaximum coding unit and prediction units of a partition type 2N×nDaccording to an exemplary embodiment;

FIG. 28 illustrates locations of minimum units in the 2N×2N sizedmaximum coding unit and prediction units of a partition type nL×2Naccording to an exemplary embodiment;

FIG. 29 illustrates locations of minimum units in the 2N×2N sizedmaximum coding unit and prediction units of a partition type nR×2Naccording to an exemplary embodiment;

FIG. 30 is a flowchart of a method of encoding a video based on ascanning order of hierarchical data units, according to an exemplaryembodiment; and

FIG. 31 is a flowchart of a method of decoding a video based on ascanning order of hierarchical data units, according to an exemplaryembodiment.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

Hereinafter, a video encoding apparatus, a video decoding apparatus, avideo encoding method, and a video decoding method according toexemplary embodiments will be described more fully with reference toFIGS. 1 through 31 below, in which like reference numerals refer to likeelements throughout. Video encoding and decoding based on spatiallyhierarchical data units according to exemplary embodiments will now bedescribed with reference to FIGS. 1 through 15, and video encoding anddecoding based on a scanning order of hierarchical data units accordingto exemplary embodiments will now be described with reference to FIGS.16 through 31. Expressions such as “at least one of,” when preceding alist of elements, modify the entire list of elements and do not modifythe individual elements of the list.

Hereinafter, a coding unit is an encoding data unit in which the imagedata is encoded at an encoder side and an encoded data unit in which theencoded image data is decoded at a decoder side, according to exemplaryembodiments. Also, a coded depth indicates a depth where a coding unitis encoded.

Hereinafter, an ‘image’ may denote a still image for a video or a movingimage, that is, the video itself.

FIG. 1 is a block diagram of a video encoding apparatus 100, accordingto an exemplary embodiment. Referring to FIG. 1, the video encodingapparatus 100 includes a maximum coding unit splitter 110, a coding unitdeterminer 120, and an output unit 130.

The maximum coding unit splitter 110 may split a current picture basedon a maximum coding unit for the current picture of an image. If thecurrent picture is larger than the maximum coding unit, image data ofthe current picture may be split into the at least one maximum codingunit. The maximum coding unit according to an exemplary embodiment maybe a data unit having a size of 32×32, 64×64, 128×128, 256×256, etc.,wherein a shape of the data unit is a square having a width and heightin squares of 2. The image data may be output to the coding unitdeterminer 120 according to the at least one maximum coding unit.

A coding unit according to an exemplary embodiment may be characterizedby a maximum size and a depth. The depth denotes a number of times thecoding unit is spatially split from the maximum coding unit.Accordingly, as the depth deepens or increases, deeper encoding unitsaccording to depths may be split from the maximum coding unit to aminimum coding unit. A depth of the maximum coding unit is an uppermostdepth and a depth of the minimum coding unit is a lowermost depth. Sincea size of a coding unit corresponding to each depth decreases as thedepth of the maximum coding unit deepens, a coding unit corresponding toan upper depth may include a plurality of coding units corresponding tolower depths.

As described above, the image data of the current picture is split intoone or more maximum coding units according to a maximum size of thecoding unit, and each of the maximum coding units may include deepercoding units that are split according to depths. Since the maximumcoding unit according to an exemplary embodiment is split according todepths, the image data of a spatial domain included in the maximumcoding unit may be hierarchically classified according to depths.

A maximum depth and a maximum size of a coding unit, which limit thetotal number of times a height and a width of the maximum coding unitare hierarchically split, may be predetermined.

The coding unit determiner 120 encodes at least one split regionobtained by splitting a region of the maximum coding unit according todepths, and determines a depth to output a finally encoded image dataaccording to the at least one split region. The coding unit determiner120 encodes at least one split region obtained by splitting a region ofthe maximum coding unit according to depths, and determines a depth tooutput a finally encoded image data according to the at least one splitregion. For example, the coding unit determiner 120 determines a codeddepth by encoding the image data in the deeper coding units according todepths, according to the maximum coding unit of the current picture, andselecting a depth having the least encoding errors. For example, thecoding unit determiner 120 determines a coded depth by encoding theimage data in the deeper coding units according to depths, according tothe maximum coding unit of the current picture, and selecting a depthhaving the least encoding errors. Thus, the encoded image data of thecoding unit corresponding to the determined coded depth is output by thecoding unit determiner 120. Thus, the encoded image data of the codingunit corresponding to the determined coded depth is output by the codingunit determiner 120. Also, the coding units corresponding to the codeddepth may be regarded as encoded coding units.

The determined coded depth and the encoded image data according to thedetermined coded depth are output to the output unit 130.

The image data in the maximum coding unit is encoded based on the deepercoding units corresponding to at least one depth equal to or below themaximum depth, and results of encoding the image data are compared basedon each of the deeper coding units. A depth having the least encodingerrors may be selected after comparing encoding errors of the deepercoding units. At least one coded depth may be selected for each maximumcoding unit.

The size of the maximum coding unit is split as a coding unit ishierarchically split according to depths, and as the number of codingunits increases. Also, even if coding units correspond to the same depthin one maximum coding unit, it is determined whether to split each ofthe coding units corresponding to the same depth to a lower depth bymeasuring an encoding error of the image data of each coding unit,separately. Accordingly, even when image data is included in one maximumcoding unit, the image data is split to regions according to the depthsand the encoding errors may differ according to regions in the onemaximum coding unit. Thus, the coded depths may differ according toregions in the image data. Therefore, one or more coded depths may bedetermined in one maximum coding unit, and the image data of the maximumcoding unit may be divided according to coding units of at least onecoded depth.

Accordingly, the coding unit determiner 120 may determine coding unitshaving a tree structure included in the maximum coding unit. The codingunits having a tree structure according to an exemplary embodimentinclude coding units corresponding to a depth determined to be the codeddepth, from among all deeper coding units included in the maximum codingunit. A coding unit of a coded depth may be hierarchically determinedaccording to depths in the same region of the maximum coding unit, andmay be independently determined in different regions. Similarly, a codeddepth in a current region may be independently determined from a codeddepth in another region.

A maximum depth according to an exemplary embodiment is an index relatedto a number of splitting times from a maximum coding unit to a minimumcoding unit. A first maximum depth according to an exemplary embodimentmay denote a total number of splitting times from the maximum codingunit to the minimum coding unit. A second maximum depth according to anexemplary embodiment may denote a total number of depth levels from themaximum coding unit to the minimum coding unit. For example, when adepth of the maximum coding unit is 0, a depth of a coding unit, inwhich the maximum coding unit is split once, may be set to 1, and adepth of a coding unit, in which the maximum coding unit is split twice,may be set to 2. Here, if the minimum coding unit is a coding unit inwhich the maximum coding unit is split four times, 5 depth levels ofdepths 0, 1, 2, 3 and 4 exist. In this case, the first maximum depth maybe set to 4, and the second maximum depth may be set to 5.

Prediction encoding and transformation may be performed according to themaximum coding unit. The prediction encoding and the transformation mayalso be performed based on the deeper coding units according to a depthequal to, or depths less than, the maximum depth, according to themaximum coding unit. Transformation may be performed according to amethod of orthogonal transformation or integer transformation.

Since the number of deeper coding units increases whenever the maximumcoding unit is split according to depths, encoding including theprediction encoding and the transformation may be performed on all ofthe deeper coding units generated as the depth deepens. For convenienceof description, the prediction encoding and the transformation will nowbe described based on a coding unit of a current depth, in a maximumcoding unit.

The video encoding apparatus 100 may variably select a size or shape ofa data unit for encoding the image data. In order to encode the imagedata, operations, such as prediction encoding, transformation, andentropy encoding, are performed, and at this time, the same data unitmay be used for all operations or different data units may be used foreach operation.

For example, the video encoding apparatus 100 may select not only acoding unit for encoding the image data, but also a data unit differentfrom the coding unit so as to perform the prediction encoding on theimage data in the coding unit.

In order to perform the prediction encoding in the maximum coding unit,the prediction encoding may be performed based on a coding unitcorresponding to a coded depth, i.e., based on a coding unit that is nolonger split to coding units corresponding to a lower depth.Hereinafter, the coding unit that is no longer split and becomes a basisunit for the prediction encoding will now be referred to as a predictionunit. A partition obtained by splitting the prediction unit may includea prediction unit or a data unit obtained by splitting at least one of aheight and a width of the prediction unit.

For example, when a coding unit of a size of 2N×2N (where N is apositive integer) is no longer split and becomes a prediction unit of2N×2N, a size of a partition may be 2N×2N, 2N×N, N×2N, or N×N. Examplesof a partition type include symmetrical partitions that are obtained bysymmetrically splitting a height or a width of the prediction unit,partitions obtained by asymmetrically splitting the height or the widthof the prediction unit (such as 1:n or n:1), partitions that areobtained by geometrically splitting the prediction unit, and partitionshaving arbitrary shapes.

A prediction mode of the prediction unit may be at least one of an intramode, a inter mode, and a skip mode. For example, the intra mode or theinter mode may be performed on the partition of 2N×2N, 2N×N, N×2N, orN×N. Also, the skip mode may be performed only on the partition of2N×2N. The encoding is independently performed on prediction units in acoding unit, thereby selecting a prediction mode having a least encodingerror.

The video encoding apparatus 100 may also perform the transformation onthe image data in a coding unit based not only on the coding unit forencoding the image data, but also based on a data unit that is differentfrom the coding unit.

In order to perform the transformation in the coding unit, thetransformation may be performed based on a data unit having a sizesmaller than or equal to the coding unit. For example, the data unit forthe transformation may include a data unit for an intra mode and a dataunit for an inter mode.

A data unit used as a base of the transformation will hereinafter bereferred to as a transformation unit. A transformation depth indicatinga number of splitting times to reach the transformation unit bysplitting a height and a width of the coding unit may also be set forthe transformation unit. For example, in a current coding unit of 2N×2N,a transformation depth may be 0 when a size of a transformation unit isalso 2N×2N, may be 1 when each of the height and the width of thecurrent coding unit is split into two equal parts, totally split into 4¹transformation units, and the size of the transformation unit is thusN×N, and may be 2 when each of the height and the width of the currentcoding unit is split into four equal parts, totally split into 4²transformation units, and the size of the transformation unit is thusN/2×N/2. In other words, the transformation unit may be set according toa hierarchical tree structure, in which a transformation unit of anupper transformation depth is split into four transformation units of alower transformation depth according to hierarchical characteristics ofa transformation depth.

Similar to the coding unit, the transformation unit in the coding unitmay be recursively split into smaller sized regions, so that thetransformation unit may be determined independently in units of regions.Thus, residual data in the coding unit may be divided according to thetransformation units having the tree structure according totransformation depths.

Encoding information according to coding units corresponding to a codeddepth uses not only information about the coded depth, but alsoinformation about information related to prediction encoding andtransformation. Accordingly, the coding unit determiner 120 not onlydetermines a coded depth having a minimum encoding error, but alsodetermines a partition type in a prediction unit, a prediction modeaccording to prediction units, and a size of a transformation unit fortransformation.

Coding units according to a tree structure in a maximum coding unit anda method of determining a partition, according to one or more exemplaryembodiments, will be described in detail later with reference to FIGS. 3through 12.

The coding unit determiner 120 may measure an encoding error of deepercoding units according to depths by using Rate-Distortion Optimizationbased on Lagrangian multipliers.

The output unit 130 outputs the image data of the maximum coding unit,which is encoded based on the at least one coded depth determined by thecoding unit determiner 120, and information about the encoding modeaccording to the coded depth, in bitstreams.

The encoded image data may be obtained by encoding residual data of animage. The information about the encoding mode according to coded depthmay include at least one of information about the coded depth,information about the partition type in the prediction unit, theprediction mode, and the size of the transformation unit.

The information about the coded depth may be defined by using splitinformation according to depths, which indicates whether encoding isperformed on coding units of a lower depth instead of a current depth.If the current depth of the current coding unit is the coded depth,image data in the current coding unit is encoded and output, and thusthe split information may be defined not to split the current codingunit to a lower depth. Alternatively, if the current depth of thecurrent coding unit is not the coded depth, the encoding is performed onthe coding unit of the lower depth. Thus, the split information may bedefined to split the current coding unit to obtain the coding units ofthe lower depth.

If the current depth is not the coded depth, encoding is performed onthe coding unit that is split into the coding unit of the lower depth.Since at least one coding unit of the lower depth exists in one codingunit of the current depth, the encoding is repeatedly performed on eachcoding unit of the lower depth. Thus, the encoding may be recursivelyperformed for the coding units having the same depth.

Since the coding units having a tree structure are determined for onemaximum coding unit, and information about at least one encoding mode isdetermined for a coding unit of a coded depth, information about atleast one encoding mode may be determined for one maximum coding unit.Also, a coded depth of the image data of the maximum coding unit may bedifferent according to locations since the image data is hierarchicallysplit according to depths. Thus, information about the coded depth andthe encoding mode may be set for the image data.

Accordingly, the output unit 130 may assign encoding information about acorresponding coded depth and an encoding mode to at least one of thecoding unit, the prediction unit, and a minimum unit included in themaximum coding unit.

The minimum unit according to an exemplary embodiment may be arectangular data unit obtained by splitting the minimum coding unithaving the lowermost depth by 4. Alternatively, the minimum unit may bea maximum-size rectangular data unit that may be included in all of thecoding units, prediction units, partition units, and transformationunits included in the maximum coding unit.

For example, the encoding information output through the output unit 130may be classified into encoding information according to coding units,and encoding information according to prediction units. The encodinginformation according to the coding units may include at least one ofinformation about the prediction mode and information about a size ofthe partitions. The encoding information according to the predictionunits may include at least one of information about an estimateddirection of an inter mode, information about a reference image index ofthe inter mode, information about a motion vector, information about achroma component of an intra mode, and information about aninterpolation method of the intra mode. Also, information about amaximum size of the coding unit defined according to pictures, slices,or groups of pictures (GOPs), and information about a maximum depth maybe inserted into a Sequence Parameter Set (SPS) or a header of abitstream.

In the video encoding apparatus 100, the deeper coding unit may be acoding unit obtained by dividing at least one of a height and a width ofa coding unit of an upper depth, which is one layer above, by two. Inother words, when the size of the coding unit of the current depth is2N×2N, the size of the coding unit of the lower depth may be N×N. Also,the coding unit of the current depth having the size of 2N×2N mayinclude 4 of the coding units of the lower depth.

Accordingly, the video encoding apparatus 100 may form the coding unitshaving the tree structure by determining coding units having an optimumshape and an optimum size for each maximum coding unit, based on thesize of the maximum coding unit and the maximum depth both determinedconsidering characteristics of the current picture. Also, since encodingmay be performed on each maximum coding unit by using any of variousprediction modes and transformations, an optimum encoding mode may bedetermined considering characteristics of the coding unit of variousimage sizes.

Thus, if an image having a high resolution or a large amount of data isencoded in units of related art macroblocks, a number of macroblocks perpicture excessively increases. Accordingly, a number of pieces ofcompressed information generated for each macroblock increases, and thusit is difficult to transmit the compressed information and datacompression efficiency decreases. However, by using the video encodingapparatus 100 according to an exemplary embodiment, image compressionefficiency may be increased since a coding unit is adjusted whileconsidering characteristics of an image while increasing a maximum sizeof a coding unit while considering a size of the image.

FIG. 2 is a block diagram of a video decoding apparatus 200, accordingto an exemplary embodiment. Referring to FIG. 2, the video decodingapparatus 200 includes a receiver 210, an image data and encodinginformation extractor 220, and an image data decoder 230. Definitions ofvarious terms, such as a coding unit, a depth, a prediction unit, atransformation unit, and information about various encoding modes, forvarious operations of the video decoding apparatus 200 are the same orsimilar to those described above with reference to FIG. 1 and the videoencoding apparatus 100.

The receiver 210 receives and parses a bitstream of an encoded video.The image data and encoding information extractor 220 extracts encodedimage data for each coding unit from the parsed bitstream, wherein thecoding units have a tree structure according to each maximum codingunit, and outputs the extracted image data to the image data decoder230. The image data and encoding information extractor 220 may extractinformation about a maximum size of a coding unit of a current picturefrom a header corresponding to the current picture or an SPS.

Also, the image data and encoding information extractor 220 extractsinformation about a coded depth and an encoding mode for the codingunits having a tree structure according to each maximum coding unit,from the parsed bitstream. The extracted information about the codeddepth and the encoding mode is output to the image data decoder 230.Thus, the image data in a bit stream is split into the maximum codingunit so that the image data decoder 230 decodes the image data for eachmaximum coding unit.

The information about the coded depth and the encoding mode according tothe maximum coding unit may be set for information about at least onecoding unit corresponding to the coded depth. Furthermore, theinformation about the encoding mode may include at least one ofinformation about a partition type of a corresponding coding unitcorresponding to the coded depth, information about a prediction mode,and a size of a transformation unit. Also, splitting informationaccording to depths may be extracted as the information about the codeddepth.

The information about the coded depth and the encoding mode according toeach maximum coding unit extracted by the image data and encodinginformation extractor 220 is information about a coded depth and anencoding mode determined to generate a minimum encoding error when anencoder, such as the video encoding apparatus 100, repeatedly performsencoding for each deeper coding unit according to depths according toeach maximum coding unit. Accordingly, the video decoding apparatus 200may restore an image by decoding the image data according to a codeddepth and an encoding mode that generates the minimum encoding error.

Since encoding information about the coded depth and the encoding modemay be assigned to a predetermined data unit from among a correspondingcoding unit, a prediction unit, and a minimum unit, the image data andencoding information extractor 220 may extract the information about thecoded depth and the encoding mode according to the predetermined dataunits. The predetermined data units to which the same information aboutthe coded depth and the encoding mode is assigned may be inferred to bethe data units included in the same maximum coding unit.

The image data decoder 230 restores the current picture by decoding theimage data in each maximum coding unit based on the information aboutthe coded depth and the encoding mode according to the maximum codingunits. In other words, the image data decoder 230 may decode the encodedimage data based on the extracted information about the partition type,the prediction mode, and the transformation unit for each coding unitfrom among the coding units having the tree structure included in eachmaximum coding unit. A decoding process may include at least one of aprediction including intra prediction and motion compensation, and aninverse transformation. Inverse transformation may be performedaccording to method of inverse orthogonal transformation or inverseinteger transformation.

The image data decoder 230 may perform intra prediction or motioncompensation according to a partition and a prediction mode of eachcoding unit, based on the information about the partition type and theprediction mode of the prediction unit of the coding unit according tocoded depths.

Also, the image data decoder 230 may perform inverse transformationaccording to each transformation unit in the coding unit, based on theinformation about the size of the transformation unit of the coding unitaccording to coded depths, so as to perform the inverse transformationaccording to maximum coding units.

The image data decoder 230 may determine at least one coded depth of acurrent maximum coding unit by using split information according todepths. If the split information indicates that image data is no longersplit in the current depth, the current depth is a coded depth.Accordingly, the image data decoder 230 may decode encoded data of atleast one coding unit corresponding to each coded depth in the currentmaximum coding unit by using the information about the partition type ofthe prediction unit, the prediction mode, and the size of thetransformation unit for each coding unit corresponding to the codeddepth, and output the image data of the current maximum coding unit.

In other words, data units including the encoding information includingthe same split information may be gathered by observing the encodinginformation set assigned for the predetermined data unit from among thecoding unit, the prediction unit, and the minimum unit. Moreover, thegathered data units may be considered to be one data unit to be decodedby the image data decoder 230 in the same encoding mode.

The video decoding apparatus 200 may obtain information about at leastone coding unit that generates the minimum encoding error when encodingis recursively performed for each maximum coding unit, and may use theinformation to decode the current picture. In other words, the codingunits having the tree structure determined to be the optimum codingunits in each maximum coding unit may be decoded. Also, a maximum sizeof the coding unit may be determined considering resolution and anamount of image data.

Accordingly, even if image data has a high resolution and a large amountof data, the image data may be efficiently decoded and restored by usinga size of a coding unit and an encoding mode, which are adaptivelydetermined according to characteristics of the image data, by usinginformation about an optimum encoding mode received from an encoder.

A method of determining coding units having a tree structure, aprediction unit, and a transformation unit, according to one or moreexemplary embodiments will now be described with reference to FIGS. 3through 13.

FIG. 3 is a diagram for describing a concept of coding units accordingto an exemplary embodiment. A size of a coding unit may be expressed inwidth×height, and may be 64×64, 32×32, 16×16, and 8×8, though it isunderstood that another exemplary embodiment is not limited thereto. Acoding unit of 64×64 may be split into partitions of 64×64, 64×32,32×64, or 32×32, a coding unit of 32×32 may be split into partitions of32×32, 32×16, 16×32, or 16×16, a coding unit of 16×16 may be split intopartitions of 16×16, 16×8, 8×16, or 8×8, and a coding unit of 8×8 may besplit into partitions of 8×8, 8×4, 4×8, or 4×4.

Referring to FIG. 3, first video data 310 has a resolution of 1920×1080,a maximum size of a coding unit of 64, and a maximum depth of 2. Secondvideo data 320 has a resolution of 1920×1080, a maximum size of a codingunit of 64, and a maximum depth of 3. Third video data 330 has aresolution of 352×288, a maximum size of a coding unit of 16, and amaximum depth of 1. The maximum depth shown in FIG. 3 denotes a totalnumber of splits from a maximum coding unit to a minimum decoding unit.

If a resolution is high or a data amount is large, a maximum size of acoding unit may be large so as to not only increase encoding efficiencybut also to accurately reflect characteristics of an image. Accordingly,the maximum size of the coding units of the first and second video data310 and 320 having a higher resolution than the third video data 330 maybe 64.

Since the maximum depth of the first video data 310 is 2, coding units315 of the first video data 310 may include a maximum coding unit havinga long axis size of 64, and coding units having long axis sizes of 32and 16 since depths are deepened to two layers by splitting the maximumcoding unit twice. Meanwhile, since the maximum depth of the third videodata 330 is 1, coding units 335 of the third video data 330 may includea maximum coding unit having a long axis size of 16, and coding unitshaving a long axis size of 8 since depths are deepened to one layer bysplitting the maximum coding unit once.

Since the maximum depth of the second video data 320 is 3, coding units325 of the second video data 320 may include a maximum coding unithaving a long axis size of 64, and coding units having long axis sizesof 32, 16, and 8 since the depths are deepened to 3 layers by splittingthe maximum coding unit three times. As a depth deepens (i.e.,increases), detailed information may be precisely expressed.

FIG. 4 is a block diagram of an image encoder 400 based on coding units,according to an exemplary embodiment. Referring to FIG. 4, the imageencoder 400 performs operations of the coding unit determiner 120 of thevideo encoding apparatus 100 to encode image data. For example, an intrapredictor 410 performs intra prediction on coding units in an intramode, from among a current frame 405, and a motion estimator 420 and amotion compensator 425 perform inter estimation and motion compensation,respectively, on coding units in an inter mode from among the currentframe 405 by using the current frame 405, and a reference frame 495.

Data output from the intra predictor 410, the motion estimator 420, andthe motion compensator 425 is output as a quantized transformationcoefficient through a transformer 430 and a quantizer 440. The quantizedtransformation coefficient is restored as data in a spatial domainthrough an inverse quantizer 460 and an inverse transformer 470. Therestored data in the spatial domain is output as the reference frame 495after being post-processed through a deblocking unit 480 and a loopfiltering unit 490. The quantized transformation coefficient may beoutput as a bitstream 455 through an entropy encoder 450.

In order for the image encoder 400 to be applied in the video encodingapparatus 100, elements of the image encoder 400, i.e., the intrapredictor 410, the motion estimator 420, the motion compensator 425, thetransformer 430, the quantizer 440, the entropy encoder 450, the inversequantizer 460, the inverse transformer 470, the deblocking unit 480, andthe loop filtering unit 490, perform operations based on each codingunit from among coding units having a tree structure while consideringthe maximum depth of each maximum coding unit.

Specifically, the intra predictor 410, the motion estimator 420, and themotion compensator 425 determine partitions and a prediction mode ofeach coding unit from among the coding units having a tree structurewhile considering a maximum size and a maximum depth of a currentmaximum coding unit, and the transformer 430 determines a size of thetransformation unit in each coding unit from among the coding unitshaving a tree structure.

FIG. 5 is a block diagram of an image decoder 500 based on coding units,according to an exemplary embodiment. Referring to FIG. 5, a parser 510parses encoded image data to be decoded and information about encodingused for decoding from a bitstream 505. The encoded image data is outputas inverse quantized data through an entropy decoder 520 and an inversequantizer 530, and the inverse quantized data is restored to image datain a spatial domain through an inverse transformer 540.

An intra predictor 550 performs intra prediction on coding units in anintra mode with respect to the image data in the spatial domain, and amotion compensator 560 performs motion compensation on coding units inan inter mode by using a reference frame 585.

The image data in the spatial domain, which passed through the intrapredictor 550 and the motion compensator 560, may be output as arestored frame 595 after being post-processed through a deblocking unit570 and a loop filtering unit 580. Also, the image data that ispost-processed through the deblocking unit 570 and the loop filteringunit 580 may be output as the reference frame 585.

In order to decode the image data in the image data decoder 230 of thevideo decoding apparatus 200, the image decoder 500 may performoperations that are performed after the parser 510.

In order for the image decoder 500 to be applied in the video decodingapparatus 200, elements of the image decoder 500, i.e., the parser 510,the entropy decoder 520, the inverse quantizer 530, the inversetransformer 540, the intra predictor 550, the motion compensator 560,the deblocking unit 570, and the loop filtering unit 580, performoperations based on coding units having a tree structure for eachmaximum coding unit.

Specifically, the intra predictor 550 and the motion compensator 560determine partitions and a prediction mode for each of the coding unitshaving a tree structure, and the inverse transformer 540 need todetermine a size of a transformation unit for each coding unit.

FIG. 6 is a diagram illustrating deeper coding units according todepths, and partitions, according to an exemplary embodiment. A videoencoding apparatus 100 according to an exemplary embodiment and a videodecoding apparatus 200 according to an exemplary embodiment usehierarchical coding units so as to consider characteristics of an image.A maximum height, a maximum width, and a maximum depth of coding unitsmay be adaptively determined according to the characteristics of theimage, or may be differently set by a user. Sizes of deeper coding unitsaccording to depths may be determined according to a predeterminedmaximum size of the coding unit.

Referring to FIG. 6, in a hierarchical structure 600 of coding units,according to an exemplary embodiment, the maximum height and the maximumwidth of the coding units are each 64, and the maximum depth is 4. Sincea depth deepens (i.e., increases) along a vertical axis of thehierarchical structure 600, a height and a width of the deeper codingunits are each split. Also, a prediction unit and partitions, which arebases for prediction encoding of each deeper coding unit, are shownalong a horizontal axis of the hierarchical structure 600.

For example, a first coding unit 610 is a maximum coding unit in thehierarchical structure 600, wherein a depth thereof is 0 and a size,i.e., a height by width, thereof is 64×64. The depth deepens along thevertical axis such that the hierarchical structure 600 includes a secondcoding unit 620 having a size of 32×32 and a depth of 1, a third codingunit 630 having a size of 16×16 and a depth of 2, a fourth coding unit640 having a size of 8×8 and a depth of 3, and a fifth coding unit 650having a size of 4×4 and a depth of 4. The fifth coding unit 650 havingthe size of 4×4 and the depth of 4 is a minimum coding unit.

The prediction unit and the partitions of the coding units 610, 620,630, 640, and 650 are arranged along the horizontal axis according toeach depth. In other words, if the first coding unit 610 having the sizeof 64×64 and the depth of 0 is a prediction unit, the prediction unitmay be split into partitions included in the first coding unit 610, i.e.a partition 610 having a size of 64×64, partitions 612 having a size of64×32, partitions 614 having the size of 32×64, or partitions 616 havinga size of 32×32.

Similarly, a prediction unit of the second coding unit 620 having thesize of 32×32 and the depth of 1 may be split into partitions includedin the second coding unit 620, i.e. a partition 620 having a size of32×32, partitions 622 having a size of 32×16, partitions 624 having asize of 16×32, and partitions 626 having a size of 16×16.

Similarly, a prediction unit of the third coding unit 630 having thesize of 16×16 and the depth of 2 may be split into partitions includedin the third coding unit 630, i.e., a partition having a size of 16×16included in the third coding unit 630, partitions 632 having a size of16×8, partitions 634 having a size of 8×16, and partitions 636 having asize of 8×8.

Similarly, a prediction unit of the fourth coding unit 640 having thesize of 8×8 and the depth of 3 may be split into partitions included inthe fourth coding unit 640, i.e. a partition having a size of 8×8included in the fourth coding unit 640, partitions 642 having a size of8×4, partitions 644 having a size of 4×8, and partitions 646 having asize of 4×4.

The fifth coding unit 650 having the size of 4×4 and the depth of 4 isthe minimum coding unit and a coding unit of the lowermost depth. Aprediction unit of the fifth coding unit 650 is assigned to a partitionhaving a size of 4×4. Alternatively, the prediction unit of the fifthcoding unit 650 may be split into a partition having a size of 4×4included in the fifth coding unit 650, partitions 652 each having a sizeof 4×2, partitions 654 each having a size of 2×4, and partitions 656each having a size of 2×2.

In order to determine the at least one coded depth of the coding unitsof the maximum coding unit 610, the coding unit determiner 120 of thevideo encoding apparatus 100 performs encoding for coding unitscorresponding to each depth included in the maximum coding unit 610.

A number of deeper coding units according to depths including data inthe same range and the same size increases as the depth deepens. Forexample, four coding units corresponding to a depth of 2 are required tocover data that is included in one coding unit corresponding to a depthof 1. Accordingly, in order to compare encoding results of the same dataaccording to depths, the coding unit corresponding to the depth of 1 andfour coding units corresponding to the depth of 2 are each encoded.

In order to perform encoding for a current depth from among the depths,a minimum encoding error may be selected for the current depth byperforming encoding for each prediction unit in the coding unitscorresponding to the current depth, along the horizontal axis of thehierarchical structure 600. Alternatively, the minimum encoding errormay be searched for by comparing the minimum encoding errors accordingto depths, by performing encoding for each depth as the depth deepensalong the vertical axis of the hierarchical structure 600. A depth and apartition having the minimum encoding error in the first coding unit 610may be selected as the coded depth and a partition type of the firstcoding unit 610.

FIG. 7 is a diagram for describing a relationship between a coding unit710 and transformation units 720, according to an exemplary embodiment.The video encoding apparatus 100 according to an exemplary embodimentand a video decoding apparatus 200 according to an exemplary embodimentencodes and decodes, respectively, an image according to coding unitshaving sizes smaller than or equal to a maximum coding unit for eachmaximum coding unit. Sizes of transformation units for transformationduring encoding may be selected based on data units that are not largerthan a corresponding coding unit.

Referring to FIG. 7, for example, in the video encoding apparatus 100,if a size of the coding unit 710 is 64×64, transformation may beperformed by using the transformation units 720 having a size of 32×32.

Also, data of the coding unit 710 having the size of 64×64 may beencoded by performing the transformation on each of the transformationunits having the size of 32×32, 16×16, 8×8, and 4×4, which are smallerthan 64×64, and then a transformation unit having the least codingerrors may be selected.

FIG. 8 is a diagram for describing encoding information of coding unitscorresponding to a coded depth, according to an exemplary embodiment.Referring to FIG. 8, the output unit 130 of a video encoding apparatus100 according to an exemplary embodiment may encode and transmit firstinformation 800 about a partition type, second information 810 about aprediction mode, and third information 820 about a size of atransformation unit for each coding unit corresponding to a coded depth,as information about an encoding mode.

The first information 800 indicates information about a shape of apartition obtained by splitting a prediction unit of a current codingunit, wherein the partition is a data unit for prediction encoding thecurrent coding unit. For example, a current coding unit CU_0 having asize of 2N×2N may be split into any one of a partition 802 having a sizeof 2N×2N, a partition 804 having a size of 2N×N, a partition 806 havinga size of N×2N, and a partition 808 having a size of N×N. Here, thefirst information 800 about a partition type is set to indicate one ofthe partition 804 having a size of 2N×N, the partition 806 having a sizeof N×2N, and the partition 808 having a size of N×N

The second information 810 indicates a prediction mode of eachpartition. For example, the second information 810 may indicate a modeof prediction encoding performed on a partition indicated by the firstinformation 800, i.e., an intra mode 812, an inter mode 814, or a skipmode 816.

The third information 820 indicates a transformation unit to be based onwhen transformation is performed on a current coding unit. For example,the transformation unit may be a first intra transformation unit 822, asecond intra transformation unit 824, a first inter transformation unit826, or a second intra transformation unit 828.

An image data and encoding information extractor 220 of a video decodingapparatus 200 according to an exemplary embodiment may extract and usethe information 800, 810, and 820 for decoding, according to each deepercoding unit

FIG. 9 is a diagram of deeper coding units according to depths,according to an exemplary embodiment. Split information may be used toindicate a change of a depth. The spilt information indicates whether acoding unit of a current depth is split into coding units of a lowerdepth.

Referring to FIG. 9, a prediction unit 910 for prediction encoding acoding unit 900 having a depth of 0 and a size of 2N_0×2N_0 may includepartitions of a partition type 912 having a size of 2N_0×2N_0, apartition type 914 having a size of 2N_0×N_0, a partition type 916having a size of N_0×2N_0, and a partition type 918 having a size ofN_0×N_0. FIG. 9 only illustrates the partition types 912 through 918which are obtained by symmetrically splitting the prediction unit 910,but it is understood that a partition type is not limited thereto inanother exemplary embodiment. For example, according to anotherexemplary embodiment, the partitions of the prediction unit 910 mayinclude asymmetrical partitions, partitions having a predeterminedshape, and partitions having a geometrical shape.

Prediction encoding is repeatedly performed on one partition having asize of 2N_0×2N_0, two partitions having a size of 2N_0×N_0, twopartitions having a size of N_0×2N_0, and four partitions having a sizeof N_0×N_0, according to each partition type. The prediction encoding inan intra mode and an inter mode may be performed on the partitionshaving the sizes of 2N_0×2N_0, N_0×2N_0, 2N_0×N_0, and N_0×N_0. Theprediction encoding in a skip mode is performed only on the partitionhaving the size of 2N_0×2N_0.

Errors of encoding including the prediction encoding in the partitiontypes 912 through 918 are compared, and the minimum encoding error isdetermined among the partition types. If an encoding error is smallestin one of the partition types 912 through 916, the prediction unit 910may not be split into a lower depth.

If the encoding error is the smallest in the partition type 918, a depthis changed from 0 to 1 to split the partition type 918 in operation 920,and encoding is repeatedly performed on coding units 930 having a depthof 2 and a size of N_0×N_0 to search for a minimum encoding error.

A prediction unit 940 for prediction encoding the coding unit 930 havinga depth of 1 and a size of 2N_1×2N_1 (=N_0×N_0) may include partitionsof a partition type 942 having a size of 2N_1×2N_1, a partition type 944having a size of 2N_1×N_1, a partition type 946 having a size ofN_1×2N_1, and a partition type 948 having a size of N_1×N_1.

If an encoding error is the smallest in the partition type 948, a depthis changed from 1 to 2 to split the partition type 948 in operation 950,and encoding is repeatedly performed on coding units 960, which have adepth of 2 and a size of N_2×N_2 to search for a minimum encoding error.

When a maximum depth is d, split operations according to each depth maybe performed up to when a depth becomes d−1, and split information maybe encoded up to when a depth is one of 0 to d−2. For example, whenencoding is performed up to when the depth is d−1 after a coding unitcorresponding to a depth of d−2 is split in operation 970, a predictionunit 990 for prediction encoding a coding unit 980 having a depth of d−1and a size of 2N_(d−1)×2N_(d−1) may include partitions of a partitiontype 992 having a size of 2N_(d−1)×2N_(d−1), a partition type 994 havinga size of 2N_(d−1)×N_(d−1), a partition type 996 having a size ofN_(d−1)×2N_(d−1), and a partition type 998 having a size ofN_(d−1)×N_(d−1).

Prediction encoding may be repeatedly performed on one partition havinga size of 2N_(d−1)×2N_(d−1), two partitions having a size of2N_(d−1)×N_(d−1), two partitions having a size of N_(d−1)×2N_(d−1), fourpartitions having a size of N_(d−1)×N_(d−1) from among the partitiontypes 992 through 998 to search for a partition type having a minimumencoding error.

Even when the partition type 998 has the minimum encoding error, since amaximum depth is d, a coding unit CU_(d−1) having a depth of d−1 is nolonger split to a lower depth, and a coded depth for the coding units ofa current maximum coding unit 900 is determined to be d−1 and apartition type of the current maximum coding unit 900 may be determinedto be N_(d−1)×N_(d−1). Also, since the maximum depth is d and a minimumcoding unit 980 having a lowermost depth of d−1 is no longer split to alower depth, split information for the minimum coding unit 980 is notset.

A data unit 999 may be considered a minimum unit for the current maximumcoding unit. A minimum unit according to an exemplary embodiment may bea rectangular data unit obtained by splitting a minimum coding unit 980by 4. By performing the encoding repeatedly, a video encoding apparatus100 according to an exemplary embodiment may select a depth having theminimum encoding error by comparing encoding errors according to depthsof the coding unit 900 to determine a coded depth, and set acorresponding partition type and a prediction mode as an encoding modeof the coded depth.

As such, the minimum encoding errors according to depths are compared inall of the depths of 1 through d, and a depth having the least encodingerrors may be determined as a coded depth. At least one of the codeddepth, the partition type of the prediction unit, and the predictionmode may be encoded and transmitted as information about an encodingmode. Also, since a coding unit is split from a depth of 0 to a codeddepth, only split information of the coded depth is set to 0, and splitinformation of depths excluding the coded depth are set to 1.

An image data and encoding information extractor 220 of a video decodingapparatus 200 according to an exemplary embodiment may extract and usethe information about the coded depth and the prediction unit of thecoding unit 900 to decode the partition 912. The video decodingapparatus 200 may determine a depth, in which split information is 0, asa coded depth by using split information according to depths, and useinformation about an encoding mode of the corresponding depth fordecoding.

FIGS. 10 through 12 are diagrams for describing a relationship betweencoding units 1010, prediction units 1060, and transformation units 1070,according to an exemplary embodiment.

Referring to FIGS. 10 through 12, the coding units 1010 are coding unitshaving a tree structure, corresponding to coded depths determined by avideo encoding apparatus 100 according to an exemplary embodiment, in amaximum coding unit. The prediction units 1060 are partitions ofprediction units of each of the coding units 1010, and thetransformation units 1070 are transformation units of each of the codingunits 1010.

When a depth of a maximum coding unit is 0 in the coding units 1010,depths of coding units 1012 and 1054 are 1, depths of coding units 1014,1016, 1018, 1028, 1050, and 1052 are 2, depths of coding units 1020,1022, 1024, 1026, 1030, 1032, and 1048 are 3, and depths of coding units1040, 1042, 1044, and 1046 are 4.

In the prediction units 1060, some encoding units 1014, 1016, 1022,1032, 1048, 1050, 1052, and 1054 are obtained by splitting the codingunits of the coding units 1010. For example, partition types in thecoding units 1014, 1022, 1050, and 1054 have a size of 2N×N, partitiontypes in the coding units 1016, 1048, and 1052 have a size of N×2N, anda partition type of the coding unit 1032 has a size of N×N. Predictionunits and partitions of the coding units 1010 are smaller than or equalto each coding unit.

Transformation or inverse transformation is performed on image data ofthe coding unit 1052 in the transformation units 1070 in a data unitthat is smaller than the coding unit 1052. Also, the coding units 1014,1016, 1022, 1032, 1048, 1050, and 1052 in the transformation units 1070are different from those in the prediction units 1060 in terms of sizesand shapes. For example, video encoding and decoding apparatuses 100 and200 according to exemplary embodiments may perform intra prediction,motion estimation, motion compensation, transformation, and inversetransformation individually on a data unit in the same coding unit.

Accordingly, encoding is recursively performed on each of coding unitshaving a hierarchical structure in each region of a maximum coding unitto determine an optimum coding unit, and thus coding units having arecursive tree structure may be obtained. Encoding information mayinclude at least one of split information about a coding unit,information about a partition type, information about a prediction mode,and information about a size of a transformation unit. Table 1 showsexemplary encoding information that may be set by the video encoding anddecoding apparatuses 100 and 200.

TABLE 1 Split Information 0 (Encoding on Coding Unit having Size of 2N ×2N and Current Depth of d) Size of Transformation Unit Split SplitPartition Type Information 0 Information 1 Symmetrical Asymmetrical ofof Prediction Partition Partition Transformation Transformation SplitMode Type Type Unit Unit Information 1 Intra 2N × 2N 2N × nU 2N × 2N N ×N Repeatedly Inter 2N × N 2N × nD (Symmetrical Encode Skip  N × 2N nL ×2N Type) Coding (Only  N × N nR × 2N N/2 × N/2 Units 2N × 2N)(Asymmetrical having Type) Lower Depth of d + 1

The output unit 130 of the video encoding apparatus 100 may output theencoding information about the coding units having a tree structure, andthe image data and encoding information extractor 220 of the videodecoding apparatus 200 may extract the encoding information about thecoding units having a tree structure from a received bitstream.

Split information indicates whether a current coding unit is split intocoding units of a lower depth. If split information of a current depth dis 0, a depth in which a current coding unit is no longer split into alower depth is a coded depth, and thus information about a partitiontype, prediction mode, and a size of a transformation unit may bedefined for the coded depth. If the current coding unit is further splitaccording to the split information, encoding is independently performedon four split coding units of a lower depth.

A prediction mode may be one of an intra mode, an inter mode, and a skipmode. The intra mode and the inter mode may be defined in all partitiontypes, and the skip mode may be defined only in a partition type havinga size of 2N×2N.

The information about the partition type may indicate symmetricalpartition types having sizes of 2N×2N, 2N×N, N×2N, and N×N, which areobtained by symmetrically splitting at least one of a height and a widthof a prediction unit, and asymmetrical partition types having sizes of2N×nU, 2N×nD, nL×2N, and nR×2N, which are obtained by asymmetricallysplitting at least one of the height and the width of the predictionunit. The asymmetrical partition types having the sizes of 2N×nU and2N×nD may be respectively obtained by splitting the height of theprediction unit in 1:3 and 3:1, and the asymmetrical partition typeshaving the sizes of nL×2N and nR×2N may be respectively obtained bysplitting the width of the prediction unit in 1:3 and 3:1

The size of the transformation unit may be set to be two types in theintra mode and two types in the inter mode. For example, if splitinformation of the transformation unit is 0, the size of thetransformation unit may be 2N×2N, which is the size of the currentcoding unit. If split information of the transformation unit is 1, thetransformation units may be obtained by splitting the current codingunit. Also, if a partition type of the current coding unit having thesize of 2N×2N is a symmetrical partition type, a size of atransformation unit may be N×N, and if the partition type of the currentcoding unit is an asymmetrical partition type, the size of thetransformation unit may be N/2×N/2.

The encoding information about coding units having a tree structure mayinclude at least one of a coding unit corresponding to a coded depth, aprediction unit, and a minimum unit. The coding unit corresponding tothe coded depth may include at least one of a prediction unit and aminimum unit including the same encoding information.

Accordingly, it is determined whether adjacent data units are includedin the same coding unit corresponding to the coded depth by comparingencoding information of the adjacent data units. Also, a correspondingcoding unit corresponding to a coded depth is determined by usingencoding information of a data unit, and thus a distribution of codeddepths in a maximum coding unit may be determined.

Therefore, if a current coding unit is predicted based on encodinginformation of adjacent data units, encoding information of data unitsin deeper coding units adjacent to the current coding unit may bedirectly referred to and used.

Alternatively, if a current coding unit is predicted based on encodinginformation of adjacent data units, data units adjacent to the currentcoding unit are searched using encoded information of the data units,and the searched adjacent coding units may be referred to for predictingthe current coding unit.

FIG. 13 is a diagram for describing a relationship between a codingunit, a prediction unit or a partition, and a transformation unit,according to encoding mode information of Table 1 according to anexemplary embodiment. Referring to FIG. 13, a maximum coding unit 1300includes coding units 1302, 1304, 1306, 1312, 1314, 1316, and 1318 ofcoded depths. Here, since the coding unit 1318 is a coding unit of acoded depth, split information may be set to 0. Information about apartition type of the coding unit 1318 having a size of 2N×2N may be setto be a partition type 1322 having a size of 2N×2N, a partition type1324 having a size of 2N×N, a partition type 1326 having a size of N×2N,a partition type 1328 having a size of N×N, a partition type 1332 havinga size of 2N×nU, a partition type 1334 having a size of 2N×nD, apartition type 1336 having a size of nL×2N, or a partition type 1338having a size of nR×2N.

When the partition type is set to be symmetrical, i.e., the partitiontype 1322, 1324, 1326, or 1328, a transformation unit 1342 having a sizeof 2N×2N is set if split information (TU size flag) of a transformationunit is 0, and a transformation unit 1344 having a size of N×N is set ifa TU size flag is 1.

When the partition type is set to be asymmetrical, i.e., the partitiontype 1332, 1334, 1336, or 1338, a transformation unit 1352 having a sizeof 2N×2N is set if a TU size flag is 0, and a transformation unit 1354having a size of N/2×N/2 is set if a TU size flag is 1.

Referring to FIG. 13, the TU size flag is a flag having a value or 0 or1, though it is understood that another exemplary embodiment is notlimited to a 1-bit flag, and a transformation unit may be hierarchicallysplit having a tree structure while the TU size flag increases from 0 inanother exemplary embodiment.

In this case, when the TU size flag is used together with a maximum sizeof a transformation unit and a minimum size thereof, the size of anactually used transformation unit may be represented. The video encodingapparatus 100 may encode size information of a maximum transformationunit, size information of a minimum transformation unit, and maximumsplit information of a transformation unit. The encoded size informationof the maximum transformation unit, the encoded size information of theminimum transformation unit, and the maximum split information of atransformation unit may be inserted into the SPS. The video decodingapparatus 200 may perform video decoding by using the size informationof the maximum transformation unit, the size information of the minimumtransformation unit, and the maximum split information of atransformation unit.

For example, when a current coding unit has a size of 64×64, a maximumtransformation unit has a size of 32×32, and a TU size flag is 0, atransformation unit may be set to have a size of 32×32. When the currentencoding unit has a size of 64×64, the maximum transformation unit has asize of 32×32, and the TU size flag is 1, the transformation unit may beset to have a size of 16×16. When the current encoding unit has a sizeof 64×64, the maximum transformation unit has a size of 32×32, and theTU size flag is 2, the transformation unit may be set to have a size of8×8.

For example, when the current coding unit has a size of 32×32, a minimumtransformation unit has a size of 32×32, and the TU size flag is 0, thetransformation unit may be set to have a size of 32×32. Since the sizeof the transformation unit cannot be smaller than 32×32, no TU sizeflags can be set.

For example, when the current encoding unit has a size of 64×64 andmaximum split information of a transformation unit is 1, the TU sizeflag may be 0 or 1, and other TU size flag cannot be set.

Accordingly, when the maximum split information of a transformation unitis defined as ‘MaxTransformSizeIndex’, the size of the minimumtransformation unit is defined as ‘MinTransformSize’, and the size of atransformation unit when the TU size flag is 0 is defined as‘RootTuSize’, a minimum transformation unit size possible at a currentcoding unit, ‘CurrMinTuSize’, may be defined as the following Equation(1):CurrMinTuSize=max(MinTransformSize,RootTuSize/(2{circumflex over( )}MaxTransformSizeIndex))  (1)

In comparison with the minimum transformation unit size possible at acurrent coding unit, ‘CurrMinTuSize’, the transformation unit size whenthe TU size flag is 0, ‘RootTuSize’, may represent a maximumtransformation unit size that can be employed by a system. In otherwords, according to the Equation (1), since ‘RootTuSize/(2{circumflexover ( )}MaxTransformSizeIndex)’ denotes a transformation unit sizeobtained by splitting the transformation unit size when the TU size flagis 0, ‘RootTuSize’, by a number of times corresponding to the maximumsplit information of a transformation unit, and ‘MinTransformSize’denotes a minimum transformation unit size, a smaller size from amongthe two sizes may be the minimum transformation unit size possible atthe current coding unit, ‘CurrMinTuSize’. In an exemplary embodiment,Equation (1) above and equations hereinafter are relationship equations.

The current maximum transformation unit size RootTuSize may varydepending on the type of prediction mode.

For example, when a current prediction mode is an inter mode, RootTuSizemay be determined according to Equation (2) below, where‘MaxTransformSize’ indicates a maximum transformation unit size and‘PUSize’ indicates a current prediction unit size:RootTuSize=min(MaxTransformSize,PUSize)  (2)

In other words, when the current prediction mode is an inter mode, thetransformation unit size when the TU size flag is 0, ‘RootTuSize’, maybe set to be a smaller size from among the maximum transformation unitsize and the current prediction unit size.

When a prediction mode of a current partition unit is an intra mode,‘RootTuSize’ may be determined according to Equation (3) below, where‘PartitionSize’ indicates the size of the current partition unit:RootTuSize=min(MaxTransformSize,PartitionSize)  (3)

In other words, when the current prediction mode is an intra mode, thetransformation unit size when the TU size flag is 0, ‘RootTuSize’, maybe set to be a smaller size from among the maximum transformation unitsize and the current partition unit size.

However, it should be noted that a current maximum transformation unitsize ‘RootTuSize’ that varies according to the prediction mode of apartition unit is only an exemplary, and that a factor for determiningthe current maximum transformation unit size ‘RootTuSize’ is not limitedthereto in another exemplary embodiment.

FIG. 14 is a flowchart illustrating a method of encoding a video,according to an exemplary embodiment.

Referring to FIG. 14, in operation 1210, a current picture is split intoat least one maximum coding unit. A maximum depth indicating a totalnumber of possible splitting splits may be set in advance.

In operation 1220, at least one split region obtained by splitting aregion of the maximum coding unit according to depths is encoded, sothat a coded depth to output a final encoding result is determined foreach at least one split region and a coding unit according to a treestructure is determined.

The maximum coding unit is spatially split whenever the depth deepens,and thus is split into coding units of a lower depth. Each coding unitmay be split into coding units of another lower depth by being spatiallysplit independently from adjacent coding units. Encoding is repeatedlyperformed on each coding unit according to depths.

Also, partition types and transformation units having a minimum encodingerror are determined for each deeper coding unit. In order to determinea coded depth having a minimum encoding error in each maximum codingunit, encoding errors may be measured and compared in all deeper codingunits according to depths.

In operation 1230, image data being a final result of the encoding of atleast one split region is output, with encoded information about a codeddepth and an encoding mode. The encoded information about the encodingmode may include information about a coded depth or split information,information about a partition type of a prediction unit, a predictionmode, and a size of a transformation unit. The encoded information aboutthe encoding mode may be transmitted to a decoder with the encoded imagedata.

FIG. 15 is a flowchart illustrating a method of decoding a video,according to an exemplary embodiment.

Referring to FIG. 15, in operation 1310, a bitstream of an encoded videois received and parsed.

In operation 1320, encoded image data of a current picture assigned to amaximum coding unit is acquired from the parsed bitstream, andinformation about a coded depth and an encoding mode according tomaximum coding units are extracted from the parsed bitstream. The codeddepth of each maximum coding unit is a depth having a minimum encodingerror in each maximum coding unit. In encoding each maximum coding unit,the image data is encoded based on at least one data unit obtained byhierarchically splitting each maximum coding unit according to depths.

According to the information about the coded depth and the encodingmode, the maximum coding unit may be split into coding units having atree structure. Each of the coding units having the tree structure isdetermined as a coding unit corresponding to a coded depth, and isoptimally encoded as to output the minimum encoding error. Accordingly,encoding and decoding efficiency of an image may be improved by decodingeach piece of encoded image data in the coding units having a treestructure after determining at least one coded depth according to codingunits.

In operation 1330, the encoded image data of each maximum coding unit isdecoded based on the information about a coded depth and an encodingmode according to maximum coding units. The decoded image data may bereproduced by a reproducing apparatus, stored in a storage medium, ortransmitted through a network according to one or more exemplaryembodiments.

A method and apparatus for encoding a video based on a scanning order ofhierarchical data units and a method and apparatus for decoding a videobased on a scanning order of hierarchical data units according toexemplary embodiments will now be described in detail with reference toFIGS. 16 to 31.

FIG. 16 is a block diagram of a video encoding apparatus 1400 based on ascanning order of hierarchical data units, according to an exemplaryembodiment.

The video encoding apparatus 1400 based on the scanning order ofhierarchical data units includes a maximum coding unit splitter 1410, acoding unit determiner 1420, and an output unit 1430. Since the videoencoding apparatus 1400 based on the scanning order of hierarchical dataunits is an exemplary embodiment of the video encoding apparatus 100,the maximum coding unit splitter 110, the coding unit determiner 120,and the output unit 130 of the video encoding apparatus 100 maycorrespond to the maximum coding unit splitter 1410, the coding unitdeterminer 1420, and the output unit 1430, respectively.

The maximum coding unit splitter 1410 splits a picture of an input imageinto maximum coding units each having a predetermined size, and imagedata of each maximum coding unit is output to the coding unit determiner1420.

The coding unit determiner 1420 hierarchically splits the regions of themaximum coding units received from the maximum coding unit splitter 1410as the depth deepens, and performs encoding based on deeper coding unitsaccording to depths corresponding to numbers of splitting timesindependently on the hierarchically split regions.

Encoding based on each deeper coding unit includes prediction based onprediction units of shapes and sizes smaller or equal to the deepercoding unit, and transformation based on transformation units of sizessmaller than or equal to the deeper coding unit.

Accordingly, the coding unit determiner 1420 may determine at least onecoded depth where a result of the encoding is to be output, thepartition type of a coding unit having the coded depth, a predictionmode, the size of a transformation unit, etc. Pieces of informationassociated with the coding unit having the coded depth determined inthis way may be determined as information about the encoding mode.

To determine, for each independent region of a maximum coding unit, acoded depth where the encoding result is to be output and a coding modeassociated with the coded depth, the coding unit determiner 1420 maysearch for at least one coded depth of coding units in which the fewestencoding errors occur, from among the deeper coding units according tothe depths.

According to an exemplary embodiment, the order of encoding the maximumcoding units is a raster scanning order. Coding units according tohierarchical depths in a maximum coding unit may be scanned in thezigzag scanning order among deeper coding units corresponding to thesame depth. The order of encoding the minimum units included in amaximum coding unit may be a zigzag scanning order or a raster scanningorder.

A relationship between an upper coding unit and a lower coding unit fromamong the deeper coding units included in the maximum coding unitaccording to the zigzag scanning order will now be illustrated. Arelative location of a predetermined lower coding unit with respect toan upper coding unit may be represented as an index of the predeterminedlower coding unit based on the zigzag scanning order of lower codingunits included in the upper coding unit.

Hereinafter, the index of a lower data unit with respect to an upperdata unit is interpreted as representing a scanning order based on thezigzag scanning order between lower data units existing on the samelevel in the upper data unit.

For example, the coding unit determiner 1420 may use the index of acoding unit of a predetermined depth within a maximum coding unit inorder to represent relative locations of coding units of a predetermineddepth that are included in the maximum coding unit. The coding unitdeterminer 1420 may use the indices of the minimum units of the maximumcoding unit in order to represent the relative locations of the minimumunits with respect to the maximum coding unit.

The coding unit determiner 1420 may transform a relative location of alower data unit defined according to a zigzag scanning order withrespect to an upper data unit into an absolute location of the lowerdata unit on a current picture. Furthermore, the coding unit determiner1420 may transform the absolute location of the lower data unit on thecurrent picture into the relative location of the lower data unit withrespect to the upper data unit defined according to the zigzag scanningorder. An example of the absolute location of a predetermined data unitmay be a combination of an x coordinate value and a y coordinate valueof a pixel on the left upper end of the predetermined data unit on thecurrent picture.

Relative locations between the upper coding units and the lower codingunits included in a maximum coding unit may be transformed into eachother, based on the hierarchical structure between data units such as amaximum coding unit, deeper coding units, a coding unit of a codeddepth, a prediction unit, a minimum unit, and the like.

The coding unit determiner 1420 may determine the relative location of alower coding unit with respect to a current maximum coding unit by usingthe relative location of an upper coding unit with respect to thecurrent maximum coding unit. The coding unit determiner 1420 may alsodetermine the absolute location of a lower coding unit of a currentpicture by using the relative location of the upper coding unit withrespect to the current maximum coding unit.

The coding unit determiner 1420 may also determine the relative locationof the lower coding unit of the current maximum coding unit by using theabsolute location of an upper coding unit on the current picture. Thecoding unit determiner 1420 may also determine the absolute location ofthe lower coding unit of the current picture by using the absolutelocation of the upper coding unit on the current picture.

The coding unit determiner 1420 may use the index of a minimum unitlocated on the left upper end of each deeper coding unit included in acurrent maximum coding unit with respect to the current maximum codingunit, in order to define a relationship between an upper coding unit anda lower coding unit from among the deeper coding units. For example, thecoding unit determiner 1420 may determine the index of a minimum unitlocated on the left upper end of a predetermined lower coding unitincluded in an upper coding unit with respect to a current maximumcoding unit, by using the index of a minimum unit located on the leftupper end of an upper coding unit with respect to the current maximumcoding unit.

The coding unit determiner 1420 may determine the relative locations ofthe minimum units included in a current prediction unit with respect tothe current maximum coding unit by using the relative location of apredetermined coding unit included in the current prediction unit withrespect to the current maximum coding unit.

For example, the coding unit determiner 1420 may determine the index ofa minimum unit located on the right upper end of the current predictionunit with respect to the current maximum coding unit or the index of aminimum unit located on the left lower end of the current predictionunit with respect to the current maximum coding unit, by using the indexof a minimum unit located on the left upper end of the currentprediction unit.

The coding unit determiner 1420 may determine a method of determining anindex with respect to the current maximum coding unit, based on thepartition type and partition index of a coding unit of a coded depththat includes a prediction unit. The partition type may be a symmetricalpartition type, such as 2N×2N, 2N×N, N×2N, or N×N, for halving theheight or width of a coding unit of a coded depth, or an asymmetricalpartition type for asymmetrically splitting the height or width of thecoding unit of the coded depth.

Accordingly, the coding unit determiner 1420 may determine the indicesof the minimum units included in the current prediction unit withrespect to the current maximum coding unit, by using the index of theminimum unit located on the left upper end of the current predictionunit with respect to the current maximum coding unit and the size, thepartition type, and the partition index of the coding unit of the codeddepth that includes a prediction unit.

The coding unit determiner 1420 may determine the absolute location of aminimum unit on the left upper end of a current coding unit, based onthe relative location of the current coding unit with respect to thecurrent maximum coding unit.

The coding unit determiner 1420 may determine the absolute location ofthe minimum unit on the left upper end of the current prediction unit,based on the relative location of a coding unit of the currentprediction unit with respect to the current maximum coding unit.

The coding unit determiner 1420 may refer to neighborhood information ofa current data unit in order to encode image data of the current dataunit. For example, neighborhood information of an adjacent maximumcoding unit, an adjacent coding unit, and the like may be referred to inorder to perform prediction encoding on the current maximum coding unit,the current coding unit, or the current prediction unit.

In detail, the neighborhood information of the current maximum codingunit may include a maximum coding unit, a prediction unit, and a minimumunit located on the left side of the current maximum coding unit, amaximum coding unit, a prediction unit, and a minimum unit located onthe upper side thereof, a maximum coding unit, a prediction unit, and aminimum unit located on the right upper side thereof, and a maximumcoding unit, a prediction unit, and a minimum unit located on the leftupper side thereof.

The coding unit determiner 1420 may determine the index of an adjacentminimum unit of the current prediction unit with respect to acorresponding maximum coding unit and usability indicating whether theadjacent minimum unit can be used as reference information, by using theindices of the minimum units included in the current prediction unit, inorder to determine reference information for arbitrary directive intraprediction.

For example, the coding unit determiner 1420 may determine whether aprediction unit including the adjacent minimum unit deviates from acurrent picture, by using the size of the prediction unit based on theabsolute location of the current maximum coding unit, the absolutelocation of the current coding unit, and the partition type and relativelocation of the current prediction unit. If the adjacent minimum unit oran adjacent prediction unit deviates from the current picture or if theadjacent minimum unit or the adjacent prediction unit does not deviatefrom the current picture but is not yet accessible data according to thezigzag scanning order, it may be determined that the correspondingneighboring information cannot be used as the reference information.

The coding unit determiner 1420 may determine information about theupper data unit, based on information about the lower data unit. Forexample, since encoding information including at least one ofinformation about a coding unit of a corresponding depth, informationabout splitting of the coding unit of the corresponding depth intoprediction units, and information about a prediction mode of theprediction units is set for a minimum unit, information regarding acoding unit, a prediction unit, and a maximum coding unit including theminimum unit may be inferred.

The information about the coded depth and the encoding mode associatedwith the coded depth, determined by the coding unit determiner 1420, areoutput to the output unit 1430.

The output unit 1430 outputs all information about a coded depth and anencoding mode associated with the coded depth for each maximum codingunit, and encoded video data being a result of the encoding.

FIG. 17 is a block diagram of a video decoding apparatus 1500 based on ascanning order of hierarchical data units, according to an exemplaryembodiment.

Referring to FIG. 17, the video decoding apparatus 1500 based on thescanning order of hierarchical data units includes a receiver 1510, anextractor 1520, and a decoder 1530. The video decoding apparatus 1500based on the scanning order of hierarchical data units may be anexemplary embodiment of the video decoding apparatus 200 of FIG. 2. Thereceiver 210, the image data and encoding information extractor 220, andthe image data decoder 230 of the video decoding apparatus 200 maycorrespond to the receiver 1510, the extractor 1520, and the decoder1530, respectively, of the video decoding apparatus 1500.

The receiver 1510 receives and parses a bitstream for an encoded video.

The extractor 1510 receives the parsed bitstream from the receiver 1520and extracts information about a coded depth and an encoding modeassociated with the coded depth for each maximum coding unit, andencoded video data from the parsed bitstream.

The decoder 1530 receives the information about a coded depth and anencoding mode and the encoded video data from the extractor 1520 anddecodes video data encoded for each coding unit of at least one codeddepth based on the information about a coded depth and an encoding mode.

Video data restored by the decoding may be transmitted to variousreproducible terminals or stored in storage apparatuses.

According to an exemplary embodiment, the order of encoding maximumcoding units is a raster scanning order. Coding units according tohierarchical depths included in a maximum coding unit may be scanned ina zigzag scanning order among coding units having the same depth. Theorder of encoding the minimum units included in a maximum coding unitmay be a zigzag scanning order or a raster scanning order.

Similar to the coding unit determiner 1420, the decoder 1530 maymutually transform and search for the absolute locations and relativelocations of data units, based on the order of scanning the hierarchicaldata units.

For example, the decoder 1530 may determine the absolute or relativelocation of a lower coding unit by using the relative or absolutelocation of an upper coding unit. The decoder 1530 may determine theindex of a minimum unit located on the left upper end of a predeterminedlower coding unit included in the upper coding unit with respect to thecurrent maximum coding unit, by using the index of a minimum unitlocated on the left upper end of the upper coding unit with respect tothe current maximum coding unit.

The decoder 1530 may determine the relative locations of the minimumunits included in the current prediction unit with respect to thecurrent maximum coding unit by using the relative location of apredetermined minimum unit included in the current prediction unit withrespect to the current maximum coding unit. For example, the decoder1530 may determine the index of a minimum unit located on the rightupper end of the current prediction unit with respect to the currentmaximum coding unit or the index of a minimum unit located on the leftlower end of the current prediction unit with respect to the currentmaximum coding unit, by using the index of a minimum unit located on theleft upper end of the current prediction unit.

The decoder 1530 may determine a method of determining the indices ofthe minimum units of the current prediction unit with respect to thecurrent maximum coding unit, according to the partition type andpartition index of a coding unit of a coded depth that includes aprediction unit.

The decoder 1530 may determine the absolute location of a minimum uniton the left upper end of the current coding unit, based on the relativelocation of the current coding unit with respect to the current maximumcoding unit. The decoder 1530 may determine the absolute location of aminimum unit on the left upper end of the current prediction unit, basedon the relative location of the current prediction unit with respect tothe current maximum coding unit.

The decoder 1530 may analyze the index of an adjacent minimum unit ofthe current prediction unit with respect to a corresponding maximumcoding unit, the usability of the adjacent minimum unit, and the addressof an adjacent maximum coding unit, by using the indices of the minimumunits included in the current prediction unit according to the zigzagscanning order, in order to determine the reference information forarbitrary directive intra prediction.

For example, the decoder 1530 may determine whether a prediction unitincluding the adjacent minimum unit deviates from the current picture,by using the size of the prediction unit based on the absolute locationof the current maximum coding unit, the absolute location of the currentcoding unit, and the partition type and relative location of the currentprediction unit.

Since encoding information including at least one of information about acoding unit of a corresponding depth, information about splitting of thecoding unit of the corresponding depth into prediction units, andinformation about a prediction mode of the prediction units may beextracted for a minimum unit, the decoder 1530 may predict and useinformation about a coding unit of the corresponding coded depth,prediction units, and a maximum coding unit including the minimum unitbased on the encoding information about the minimum unit.

Accordingly, the video encoding apparatus 1400 based on the scanningorder of hierarchical data units and the video decoding apparatus 1500based on the scanning order of hierarchical data units are capable ofaccurately and quickly analyzing the locations of various types of dataunits when using data units relatively larger than existing macroblocksand the various types of data units, by mutually transforming and usingthe relative locations and absolute locations of the data units based onthe scanning order of the hierarchical data units, thereby efficientlyencoding or decoding a video.

FIG. 18 illustrates a relationship between the locations and scanindices of a slice, a coding unit, a prediction unit, a partition, and atransformation unit according to a hierarchical structure according toan exemplary embodiment.

Referring to FIG. 18, a picture is split into slices, and a slice 1850includes a plurality of maximum coding units LCUs. A location lcuAddr ofa maximum coding unit 1860 from among the maximum coding units LCUsincluded in the slice 1850 may be expressed as a relative location of asample on the left upper end of the maximum coding unit 1860 compared toa sample on the left upper end of the slice 1850.

A location of a coding unit 1870 from among coding units having a treestructure in the maximum coding unit 1860 may be expressed as a scanindex cuIdx of the coding unit 1870 compared to the sample on the leftupper end of the maximum coding unit 1860. If the coding unit 1870 is acoding unit corresponding to a coded depth, that is, a coding unit thatis no longer split to a lower depth, the coding unit 1870 becomes aprediction unit 1870, and the location of the prediction unit 1870 maybe expressed as a scan index puIdx of the prediction unit 1870 comparedto the sample on the left upper end of the maximum coding unit 1860.

The prediction unit 1870 may be split into at least one PU partition. Alocation of a PU partition 1890 from among the PU partitions of theprediction unit 1870 may be expressed as a scan index puPartIdx of thePU partition 1890 compared to the sample on the left upper end of theprediction unit 1870. The prediction unit 1870 may include at leasttransformation units TUs. A location of a transformation unit 1880 fromamong the transformation units of the prediction unit 1870 may beexpressed as a scan index tuIdx of the transformation unit 1880 comparedto the sample on the left upper end of the prediction unit 1870.

The video encoding apparatus 1400 may use the locations and scan indicesof a coding unit, a prediction unit, a partition, and a transformationunit according to an exemplary embodiment described above with referenceto FIG. 18, in order to perform video encoding. The video decodingapparatus 1500 may decode encoded data of an image based on coding unitshaving a tree structure, by using the locations and scan indices of acoding unit, a prediction unit, a partition, and a transformation unit.

FIG. 19 is a diagram illustrating indices based on a zigzag scanningorder of hierarchical coded units according to an exemplary embodiment.

The coding unit determiner 1420 of the video encoding apparatus 1400based on the scanning order of hierarchical data units and the decoder1530 of the video decoding apparatus 1500 based on the scanning order ofhierarchical data units may scan the lower data units included in anupper data unit according to the zigzag scanning order.

A maximum coding unit LCU 1600 includes coding units of a depth d−1 thatincludes a coding unit CU_d−1 1610. The depth d−1 and a depth d denotean upper depth and a lower depth, respectively, and the coding unitCU_d−1 1610 of the depth d−1 and coding units CU_d 1630, 1640, 1650, and1660 of the depth d are an upper coding unit and lower coding units,respectively.

The upper coding unit CU_d−1 1610 may include the four coding units CU_d1630, 1640, 1650, and 1660 as lower coding units. In particular, thefour lower coding units CU_d 1630, 1640, 1650, and 1660 may be scannedin the order of the coding unit CU_d 1630 having a coding unit index of0, the coding unit CU_d 1640 having a coding unit index of 1, the codingunit CU_d 1650 having a coding unit index of 2, and the coding unit CU_d1660 having a coding unit index of 3, according to indices representingthe zigzag scanning order.

The coding unit determiner 1420 and the decoder 1530 may determine arelative location of a minimum unit SU_d located on the left upper endof each of the lower coding units CU_d 1630, 1640, 1650, and 1660 withrespect to a maximum coding unit by using the relative location of aminimum unit SU_d−1 1620 located on the left upper end of the uppercoding unit CU_d−1 1610 with respect to the maximum coding unit.

For example, a minimum unit index suIdx_d−1 is set for the minimum unitSU_d−1 1620 located on the left upper end of the upper coding unitCU_d−1 1610, according to a zigzag scanning method for a maximum codingunit. In this case, an index suIdx_d of the minimum unit SU_d 1670located on the left upper end of the lower coding unit CU_d 1640 of theupper coding unit CU_d−1 1610 with respect to a maximum coding unit maybe determined from the minimum unit index suIdx_d−1 of the minimum unitSU_d−1 1620 located on the left upper end of the upper coding unitCU_d−1 1610.

For convenience of explanation, a case where the upper coding unitCU_d−1 1610 having a 2N×2N size is split only into coding units CU_d1630, 1640,1650 and 1660, each having a N×N size, is assumed. An exampleof a method of determining the index suIdx_d of the minimum unit SU_d1670 located on the left upper end of the lower coding unit CU_d 1640follows Equation (4) below:suIdx_d=suIdx_d−1+cuIdx_d*NumSusInLcu/(4{circumflex over ( )}d)  (4)

where NumSusInLcu denotes the total number of minimum units SU includedin the maximum coding unit LCU 1600. Accordingly, the coding unitdeterminer 1420 and the decoder 1530 may determine the index suIdx_d ofthe minimum unit SU_d located on the left upper end of the lower codingunit CU_d, by using the index cuIdx_d of the lower coding unit CU_d andthe index suIdx_d−1 of the minimum unit SU_d−1 located on the left upperend of the upper coding unit CU_d−1.

Although Equation (4) presupposes a case where the upper coding unitCU_d−1 having a 2N×2N size is split only into the lower coding unitsCU_d each having a N×N size and illustrates only indices of coding unitswith respect to a maximum coding unit based on zigzag scanning as therelative locations of deeper coding units according to depths forconvenience of explanation, the coding unit determiner 1420 and thedecoder 1530 are not limited thereto.

The coding unit determiner 1420 and the decoder 1530 may determine therelative location of the minimum unit SU_d located on the left upper endof the lower coding unit CU_d with respect to the maximum coding unitLCU, by using the relative location of the lower coding unit CU_d withrespect to the maximum coding unit LCU and the relative location of theminimum unit SU_d−1 located on the left upper end of the lower codingunit CU_d−1 with respect to the maximum coding unit LCU.

FIG. 20 is a diagram illustrating relative locations of maximum codingunits, hierarchical coded units, and minimum units according to anexemplary embodiment.

Maximum coding units 1710, 1720, and 1730 of depth 0 exist, and a size1740 of each minimum unit corresponds to a size of a coding unit havingdepth 3. Based on the hierarchical structure of a maximum coding unit, aprediction unit, and a minimum unit, a relative location of theprediction unit with respect to the maximum coding unit follows thezigzag scanning order, and a relative location of the minimum unit withrespect to the prediction unit also follows the zigzag scanning order. Arelative location of the minimum unit with respect to the maximum codingunit also follows the zigzag scanning order.

Accordingly, relative locations of the minimum units of a currentprediction unit 1750 of the current maximum coding unit 1710 withrespect to the current maximum coding unit 1710 may be expressed asindices based on the zigzag scanning order in the current maximum codingunit 1710.

The coding unit determiner 1420 and the decoder 1530 may determine therelative locations of minimum units at a predetermined location on thecurrent prediction unit 1750 with respect to a maximum coding unit byusing the relative location of a minimum unit 1752 located on the leftupper end of the current prediction unit 1750 with respect to themaximum coding unit.

For example, the coding unit determiner 1420 and the decoder 1530 maydetermine the index of a minimum unit 1754 located on the right upperend on the current prediction unit 1750 with respect to the maximumcoding unit and the index of a minimum unit 1756 located on the leftlower end of the current prediction unit 1750 with respect to themaximum coding unit, by using the index of the minimum unit 1752 locatedon the left upper end of the current prediction unit 1750 with respectto the maximum coding unit.

In detail, the coding unit determiner 1420 and the decoder 1530 maydetermine the index of the minimum unit 1754 located on the right upperend of the current prediction unit 1750 with respect to the maximumcoding unit and the index of the minimum unit 1756 located on the leftlower end of the current prediction unit 1750 with respect to themaximum coding unit, by using the index of the minimum unit 1752 on theleft upper end of the current prediction unit 1750 with respect to themaximum coding unit, the size of a current coding unit, the partitiontype of the current coding unit, and the index of a current partitiontype. The partition type of the current coding unit indicates asplitting method and a splitting type of splitting into a currentprediction unit in the current coding unit, and the index of the currentpartition type indicates the index of the current prediction unit withrespect to the current coding unit.

According to a hierarchical structure of coding units and data units invideo encoding and video decoding according to exemplary embodiments,examples of the partition type of a coding unit, that is, the shape of aprediction unit, may include a symmetrical partition type and anasymmetrical partition type. Examples of the symmetrical partition typeof a 2N×2N size coding unit may include a 2N×2N type, a 2N×N type, aN×2N type, and a N×N type. Examples of the asymmetrical partition typemay include a 2N×nU type obtained by splitting the height of the 2N×2Nsize coding unit at a 1:3 ratio, a 2N×nU type obtained by splitting theheight of the 2N×2N size coding unit at a 3:1 ratio, a nL×2N typeobtained by splitting the width of the 2N×2N size coding unit at a 1:3ratio, and an nR×2N type obtained by splitting the width of the 2N×2Nsize coding unit at a 3:1 ratio.

The index of a minimum unit of the current prediction unit may bedetermined by using the height or width of the current prediction unitand the number of prediction units to serve as variables. However, sincethe height or width of the current prediction unit and the number ofprediction units are changed according to partition types, the codingunit determiner 1420 and the decoder 1530 may set a method ofdetermining the index of a minimum unit of the current prediction unit,according to the partition type of the current prediction unit.

Accordingly, a method of determining an index suIdxAR of the minimumunit 1754 on the right upper end of the current prediction unit 1750with respect to the maximum coding unit and an index suIdxBL of theminimum unit 1756 on the left lower end of the current prediction unit1750 with respect to the maximum coding unit by using an index suIdxALof the minimum unit 1752 on the left upper end of the current predictionunit 1750 with respect to the maximum coding unit may be selectivelydetermined according to whether the partition type of the currentprediction unit 1750 is the 2N×2N type, the 2N×N type, the N×2N type,the N×N type, the 2N×nU type, the 2N×nD type, the nL×2N type, or thenR×2N type. A method of determining the index of a minimum unit for eachpartition type of a prediction unit will be described later withreference to FIG. 22. External minimum units A 1760, B 1762, C 1764, D1766, and E 1768 adjacent to the minimum units of the current predictionunit 1750 of the current maximum coding unit 1710 are displayed asneighborhood information of the current prediction unit 1750.

For example, left neighborhood information, upper neighborhoodinformation, right upper neighborhood information, left upperneighborhood information, and left lower neighborhood information of thecurrent prediction unit 1750 may indicate the external minimum units A1760, B 1762, C 1764, D 1766, and E 1768, respectively, located outsidethe current prediction unit 1750 and on the left side, the upper side,the right upper side, the left upper side, and the left lower side ofthe minimum unit 1752 on the left upper end of the current predictionunit 1750.

The coding unit determiner 1420 and the decoder 1530 may check alocation and usability of a minimum unit that is adjacent to a currentprediction unit, and a location of a maximum coding unit. The locationof the adjacent minimum unit or the location of the maximum coding unitmay be expressed as an index of the minimum unit adjacent to the currentprediction unit or an address of an adjacent maximum coding unit thatincludes the minimum unit adjacent to the current prediction unit.

In a case other than a case (i) where the adjacent maximum coding unitis not included in a current picture, a case (ii) where the adjacentmaximum coding unit is not included in a current slice, a case (iii)where an address of the adjacent maximum coding unit is after that of acurrent maximum coding unit in terms of a scanning order, or a case (iv)where an index of a minimum unit on a left upper side of a deeper codingunit adjacent to the current prediction unit according to a zigzagscanning order is after that of a current minimum unit according to azigzag scanning order in terms of a scanning order, the data of theadjacent deeper coding unit may be found usable.

To search for neighborhood information of the current prediction unit,an index of a minimum unit located on the left upper end, the rightupper end, or the left lower end of the current prediction unit may beconsidered, and partition type information and information about acurrent depth are used. If all prediction units are not the same insize, indices of the prediction units for a current coding unit areused.

FIG. 21 illustrates adjacent prediction units for arbitrary directiveintra prediction according to an exemplary embodiment.

Referring to FIGS. 20 and 21, a method of determining the usability ofneighborhood information according to an exemplary embodiment will nowbe described. A current prediction unit CurrPU 1820 of a coding unit1810 having a coded depth in a current maximum coding unit 1800 mayrefer to neighborhood information in order to perform predictiveencoding. The coding unit determiner 1420 and the decoder 1530 maysearch for the index and usability of a prediction unit adjacent to thecurrent prediction unit 1820.

In detail, the coding unit determiner 1420 and the decoder 1530 maydetermine the location and usability of neighborhood information of thecurrent prediction unit 1820 by using an index suIdx of a minimum unitlocated on the left upper end of the current prediction unit 1820, anindex suIdxAR of a minimum unit located on the right upper end of thecurrent prediction unit 1820, an index suIdxBL of a minimum unit locatedon the left lower end of the current prediction unit 1820, the partitiontype of the current prediction unit 1820, the depth of a current codingunit, the index of the current prediction unit 1820 with respect to thecurrent coding unit, and the like.

The neighborhood information of the current prediction unit 1820 mayinclude left neighborhood information A, upper neighborhood informationB, right upper neighborhood information C, left upper neighborhoodinformation D, and left lower neighborhood information E of the currentprediction unit 1820. The location and usability of each neighborhoodinformation may include the index and usability of a minimum unitadjacent to the current prediction unit 1820 and the address of amaximum coding unit corresponding to the adjacent minimum unit.

In particular, when the coding unit determiner 1420 and the decoder 1530determine the usability of the right upper neighborhood information Cand the left lower neighborhood information E of the current predictionunit 1820 from among the pieces of the neighborhood information, thecoding unit determiner 1420 and the decoder 1530 first determine whetherthe right upper neighborhood information C and the left lowerneighborhood information E are accessible data.

The coding unit determiner 1420 and the decoder 1530 may determine theusability of a prediction unit 1850 located on the right upper side ofthe current prediction unit 1820 from among the right upper neighborhoodinformation C of the current prediction unit 1820.

By using the x coordinate value of the current maximum coding unit, thex coordinate value of the current coding unit, and the width of thecurrent prediction unit 1820 based on its partition type, if theprediction unit 1850 located on the right upper side of the currentprediction unit 1820 deviates from a current picture, the predictionunit 1850 is determined as neighborhood information that is notavailable as reference information. At this time, the x coordinate valueof the current coding unit may be determined from the index suIdxAR ofthe minimum unit on the right upper end of the current prediction unit1820. The x coordinate value of the current maximum coding unit may bedetermined from information about the current maximum coding unit basedon the index suIdx of the minimum unit on the left upper end of thecurrent prediction unit 1820.

Similarly, the coding unit determiner 1420 and the decoder 1530 maydetermine the usability of a prediction unit 1840 located on the leftupper side of the current prediction unit 1820 from among the left lowerneighborhood information E of the current prediction unit 1820. By usingthe y coordinate value of the current maximum coding unit, the ycoordinate value of the current coding unit, and the height of thecurrent prediction unit 1820 based on its partition type, if theprediction unit 1840 located on the left lower side of the currentprediction unit 1820 does not deviate from the current picture, theprediction unit 1840 is determined as neighborhood information that isavailable as reference information.

At this time, the y coordinate value of the current coding unit may bedetermined from the index suIdxBL of a minimum unit on the right upperend of the current prediction unit 1820. The y coordinate value of thecurrent maximum coding unit may be determined from the information aboutthe current maximum coding unit based on the index suIdx of the minimumunit on the left upper end of the current prediction unit 1820.

However, if the coding unit determiner 1420 and the decoder 1530ascertains, from information of a minimum unit on the left upper end ofthe prediction unit 1840 located on the right lower side of the currentprediction unit 1820, that a corresponding maximum coding unit 1830 isnot yet accessible according to the zigzag scanning order, the codingunit determiner 1420 and the decoder 1530 may determine the predictionunit 1840 located on the right lower side of the current prediction unit1820 to be neighborhood information that is not available as referenceinformation.

The coding unit determiner 1420 and the decoder 1530 may determine theindex and usability of an adjacent minimum unit and search for the indexand usability of a corresponding adjacent prediction unit or the addressof a corresponding adjacent maximum coding unit from information aboutthe encoding mode of the adjacent minimum unit.

A method in which the coding unit determiner 1420 and the decoder 1530determine indices of the minimum units of a prediction unit with respectto a maximum coding unit according to the partition types of theprediction unit will now be described with reference to FIGS. 22 through29. For convenience of explanation, a case where a 2N×2N size of maximumcoding unit includes 16 minimum units and a coding unit of a currentdepth is a maximum coding unit will be illustrated. However, thestructure of a maximum coding unit, a coding unit, and a minimum unit isnot limited to the above-described case.

FIG. 22 illustrates locations of minimum units in a 2N×2N sized maximumcoding unit 1900 and in a prediction unit 1990 of a partition type 2N×2Naccording to an exemplary embodiment.

The 2N×2N-sized maximum coding unit 1900 includes 16 minimum units 1905,1910, 1915, 1920, 1925, 1930, 1935, 1940, 1945, 1950, 1955, 1960, 1965,1970, 1975, and 1980.

An index of the minimum unit 1905 on the left upper end of theprediction unit 1990 with respect to the maximum coding unit 1900, thesize of a coding unit, the partition type of the prediction unit 1990,and the partition index of the prediction unit 1990 may be used todetermine indices of the minimum units 1920 and 1965 on the right upperend and left lower end of the prediction unit 1990, respectively, withrespect to the maximum coding unit 1900.

Hereinafter, equations for determining the indices of the minimum units1905, 1920, and 1965 on the left upper end, right upper end, and leftlower end, respectively, of the prediction unit 1990 of the partitiontype 2N×2N with respect to the maximum coding unit 1900 are illustrated.An absolute index SuIdxUL of the minimum unit 1905 on the left upper endof the maximum coding unit 1900, a size SuSize of each minimum unit, andthe number LcuWidthInSus of minimum units arranged in the widthdirection of the maximum coding unit 1900 may be fixed parameters.

First, a difference between an absolute index of a minimum unit based ona raster scanning order and a relative index of a minimum unit based ona zigzag scanning order according to an exemplary embodiment will beexplained. The absolute index of a minimum unit according to anexemplary embodiment denotes an index based on a raster scanning orderof the minimum units of a maximum coding unit. The relative index of aminimum unit according to an exemplary embodiment denotes an index basedon a zigzag scanning order of the minimum units of the maximum codingunit.

The zigzag scanning order of minimum units according to an exemplaryembodiment denotes an order in which four adjacent minimum units arescanned from the left upper end, to the right upper end, to the leftlower end, and to the light lower end. Adjacent groups each including 4minimum units are scanned in a raster scanning order. For example, when8 minimum units 1905, 1910, 1915, 1920, 1925, 1930, 1935, and 1940arranged in a raster scanning order are scanned in a zigzag scanningorder, they may be scanned in the order of the minimum units 1905, 1910,1925, 1930, 1915, 1920, 1935, and 1940. Accordingly, when the absoluteindex and the relative index of the minimum unit 1905 are 0, theabsolute index of the minimum unit 1925 may be set to be 4, and therelative index thereof may be set to be 2.

In the below description of FIGS. 22 through 29, a bit operator ‘a>>b’denotes a bit shift, and denotes an operation of shifting a bitstring ato the right by b bits. When the bit operator ‘a>>b’ is described in thedecimal system, the bit operator ‘a>>b’ is the same as an operation ofdividing a value a in the decimal system by 2^(b). For example, resultsof bit operations NumSusInLcu>>1, NumSusInLcu>>2, NumSusInLcu>>3, andNumSusInLcu>>4 using the number NumSusInLcu of minimum units included ina maximum coding unit are 8(=16/2{circumflex over ( )}1),4(=16/2{circumflex over ( )}2), 2(=16/2{circumflex over ( )}3), and1(=16/2{circumflex over ( )}4), respectively. The bit operation‘NumSusInLcu>>1’ may represent an operation of transforming (increasingor decreasing) a relative index to a minimum unit that goes ahead orbehind from a current minimum unit by 8 steps according to a zigzagscanning order. Similarly, the bit operation ‘NumSusInLcu>>2’ mayrepresent an operation of transforming (increasing or decreasing) arelative index to a minimum unit that goes ahead or behind from thecurrent minimum unit by 4 steps according to the zigzag scanning order,the bit operation ‘NumSusInLcu>>3’ may represent an operation oftransforming (increasing or decreasing) a relative index to a minimumunit that goes ahead or behind from the current minimum unit by 2 stepsaccording to the zigzag scanning order, and the bit operation‘NumSusInLcu>>4’ may represent an operation of transforming (increasingor decreasing) a relative index to a minimum unit that goes ahead orbehind from the current minimum unit by one step according to the zigzagscanning order.

In the case of the prediction unit 1990 having a partition type of2N×2N, a relative index suIdxALtemp of the minimum unit 1905 on the leftupper end of the prediction unit 1990 with respect to the maximum codingunit 1900 follows the Equation (5):suIdxALtemp=ConvertAbsToRel(SuIdxAL);  (5)

where an absolute index SuIdxAL represents an index of the minimum unit1905 on the left upper end of the prediction unit 1990 with respect tothe maximum coding unit 1900. A function ConvertAbsToRel( ) converts theabsolute index of a maximum coding unit into a relative index.

In the case of the prediction unit 1990 having a partition type of2N×2N, an index SuIdxAR of the minimum unit 1920 on the right upper endof the prediction unit 1990 with respect to the maximum coding unit 1900may be determined according to the Equation (6):suIdxARtemp=ConvertAbsToRel(SuIdxUL+CuSize/SuSize−1);  (6)

According to the Equation (6), a temporary variable suIdxARtemprepresents a relative index of the minimum unit 1920 on the right upperend of the prediction unit 1990 of a partition type 2N×2N.

An index SuIdxBL of the minimum unit 1965 on the left lower end of theprediction unit 1990 with respect to the maximum coding unit 1900 may bedetermined according to the Equations (7) and (8):suIdxBLtemp=ConvertAbsToRel(SuIdxUL+((CuSize/SuSize)>>1−1)*LcuWidthInSus);  (7)

According to the Equation (7), a temporary variable suIdxBLtemprepresents a relative index of the minimum unit 1965 of the predictionunit 1990.suIdxBLtemp+=NumSusInLcu>>1;  (8)

According to the Equations (7) and (8), a temporary variable suIdxBLtemprepresents a relative index of the minimum unit 1965 of the predictionunit 1990.

Accordingly, the relative index suIdxALtemp of the minimum unit 1905 onthe left upper end of the current prediction unit 1990, the relativeindex suIdxARtemp of the minimum unit 1920 on the right upper endthereof, and the relative index suIdxBLtemp of the minimum unit 1965 onthe left lower end thereof may be determined using the index SuIdxAL ofthe minimum unit 1905 on the left upper end of the current predictionunit 1990, a size CuSize of a current coding unit, and the partitiontype and partition index of the current prediction unit 1990.

Examples of equations for determining the indices of minimum units onthe left upper end, right upper end, and left lower end of a predictionunit according to partition types 2N×N, N×2N, N×N, 2N×nU, 2N×nD, nL×2N,and nR×2N of the prediction unit on the basis of the relative indices ofthe minimum units based on the above-described equations (5), (6), and(7) will now be described in detail with reference to FIGS. 23 through29.

FIG. 23 illustrates locations of minimum units in the 2N×2N sizedmaximum coding unit 1900 and prediction units 2000 and 2010 each havinga partition type 2N×N according to an exemplary embodiment.

In the case of the prediction unit 2000 having a partition type 2N×N anda partition index 0, the Equations (5), (6), and (7) may be used withoutchanges, for indices of a minimum unit AL0 1905 on the left upper end ofthe prediction unit 2000, a minimum unit AR0 1920 on the right upper endthereof, and a minimum unit BL0 1925 on the left lower end thereof withrespect to the maximum coding unit 1900.suIdxALtemp=ConvertAbsToRel(SuIdxAL);  (5)suIdxARtemp=ConvertAbsToRel(SuIdxUL+CuSize/SuSize−1);  (6)suIdxBLtemp=ConvertAbsToRel(SuIdxUL+((CuSize/SuSize)>>1−1)*LcuWidthInSus);  (7)

In the case of the prediction unit 2010 having a partition type 2N×N anda partition index 1, Equations (9) and (10) in addition to the Equations(5), (6), (7), and (8) may be set for indices of a minimum unit AL1 1945on the left upper end of the prediction unit 2010, a minimum unit AR11960 on the right upper end thereof, and a minimum unit BL1 1965 on theleft lower end thereof.suIdxALtemp=ConvertAbsToRel(SuIdxAL);  (5)suIdxALtemp+=NumSusInLcu>>1;  (9)suIdxARtemp=ConvertAbsToRel(SuIdxUL+CuSize/SuSize−1);  (6)suIdxARtemp+=NumSusInLcu>>1;  (10)suIdxBLtemp=ConvertAbsToRel(SuIdxUL+((CuSize/SuSize)>>1−1)*LcuWidthInSus);  (7)suIdxBLtemp+=NumSusInLcu>>1;  (8)

According to the Equations (9), (10), and (8), the temporary variablessuIdxALtemp, suIdxARtemp, and suIdxBLtemp may represent the relativeindices of the minimum unit AL1 1945 on the left upper end of theprediction unit 2010, the minimum unit AR1 1960 on the right upper endthereof, and the minimum unit BL1 1965 on the left lower end thereof,respectively.

FIG. 24 illustrates locations of minimum units in the 2N×2N sizedmaximum coding unit 1900 and prediction units 2100 and 2110 each havinga partition type N×2N according to an exemplary embodiment.

In the case of the prediction unit 2100 having a partition type N×2N anda partition index 0, the Equations (5), (7), and (8) may be used withoutchanges, for indices of a minimum unit AL0 1905 on the left upper end ofthe prediction unit 2100 and a minimum unit BL0 1965 on the left lowerend thereof with respect to the maximum coding unit 1900.suIdxALtemp=ConvertAbsToRel(SuIdxAL);  (5)suIdxBLtemp=ConvertAbsToRel(SuIdxUL+((CuSize/SuSize)>>1−1)*LcuWidthInSus);  (7)suIdxBLtemp+=NumSusInLcu>>1;  (8)

An Equation (11) may be set based on the Equation (6), for the index ofa minimum unit AR0 1910 on the right upper end of the prediction unit2100 having the partition index 0 with respect to the maximum codingunit 1900:suIdxARtemp=ConvertAbsToRel(SuIdxUL+CuSize/SuSize−1);  (6)suIdxARtemp−=NumSusInLcu>>2;  (11)

According to Equations (5), (9), and (11), the temporary variablessuIdxALtemp, suIdxBLtemp, and suIdxARtemp may represent the relativeindices of the minimum unit AL0 1905 on the left upper end of theprediction unit 2100, the minimum unit BL0 1965 on the left lower endthereof, and the minimum unit AR0 1910 on the right upper end thereof,respectively.

In the case of the prediction unit 2110 having a partition type N×2N anda partition index 1, an Equation (12) may be set based on the Equation(5), for the index of a minimum unit AL1 1915 on the left upper end ofthe prediction unit 2110:suIdxALtemp=ConvertAbsToRel(SuIdxAL);  (5)suIdxALtemp+=NumSusInLcu>>2;  (12)

An Equation (13) may be set based on the Equation (7), for the index ofa minimum unit BL0 1975 on the left lower end of the prediction unit2110 having the partition type N×2N and the partition index 1 withrespect to the maximum coding unit 1900:suIdxBLtemp=ConvertAbsToRel(SuIdxUL+((CuSize/SuSize)>>1−1)*LcuWidthInSus);  (7)suIdxBLtemp+=NumSusInLcu>>2*3;  (13)

The Equation (6) may be used without changes, for the index of a minimumunit AR1 1920 on the right upper end of the prediction unit 2110 havingthe partition type N×2N and the partition index 1 with respect to themaximum coding unit 1900:suIdxARtemp=ConvertAbsToRel(SuIdxUL+CuSize/SuSize−1);  (6)

According to the Equations (12), (6), and (13), the temporary variablessuIdxALtemp, suIdxARtemp, and suIdxBLtemp may represent the relativeindices of the minimum unit AL1 1915 on the left upper end of theprediction unit 2110, the minimum unit AR1 1920 on the right upper endthereof, and the minimum unit BL1 1975 on the left lower end thereof,respectively.

FIG. 25 illustrates locations of minimum units in the 2N×2N sizedmaximum coding unit 1900 and prediction units 2200, 2210, 2220, and 2230each having a partition type N×N according to an exemplary embodiment.

The minimum units on the left upper ends of the prediction units 2200,2210, 2220, and 2230 each having a partition type N×N in the maximumcoding unit 1900 are a minimum unit AL0 1905 having a partition index 0,a minimum unit AL1 1915 having a partition index 1, a minimum unit AL21945 having a partition index 2, and a minimum unit AL3 1955 having apartition index 3, respectively. The minimum units on the right upperends of prediction units 2200, 2210, 2220, and 2230 are a minimum unitAR0 1910 having a partition index 0, a minimum unit AR1 1920 having apartition index 1, a minimum unit AR2 1950 having a partition index 2,and a minimum unit AR3 1960 having a partition index 3, respectively.The minimum units on the left lower ends of prediction units 2200, 2210,2220, and 2230 are a minimum unit BL0 1925 having a partition index 0, aminimum unit BL1 1935 having a partition index 1, a minimum unit BL21965 having a partition index 2, and a minimum unit BL3 1975 having apartition index 3, respectively.

For example, an Equation (14) may be set based on the Equation (6), forthe index of the minimum unit AR0 1910 on the right upper end of theprediction unit 2200 having the partition type N×N and the partitionindex 0 with respect to the maximum coding unit 1900:suIdxARtemp=ConvertAbsToRel(SuIdxUL+CuSize/SuSize−1);  (6)suIdxARtemp−=NumSusInLcu>>2;  (14)

For example, an Equation (15) may be set based on the Equation (5), forthe index of the minimum unit AL1 1915 on the left upper end of theprediction unit 2210 having the partition type N×N and the partitionindex 1 with respect to the maximum coding unit 1900:suIdxALtemp=ConvertAbsToRel(SuIdxAL);  (5)suIdxALtemp+=NumSusInLcu>>2;  (15)

For example, equations (7) and (16) may be set according to a partitionindex CuPartIdx, for the indices of the minimum units BL0 1925, BL11935, BL2 1965, and BL3 1975 on the left lower ends of the predictionunits 2200, 2210, 2220, and 2230 having the partition type N×N withrespect to the maximum coding unit:suIdxBLtemp=ConvertAbsToRel(SuIdxUL+((CuSize/SuSize)>>1−1)*LcuWidthInSus);  (7)suIdxBLtemp+=NumSusInLcu>>2*CuPartIdx;  (16)

Similar to the Equations (14), (15), and (16), relative indicessuIdxALtemp of the minimum units AL0 1905, AL1 1915, AL2 1945, and AL31955 on the left upper ends of the prediction units 2200, 2210, 2220,and 2230, relative indices suIdxARtemp of the minimum units AR0 1910,AR1 1920, AR2 1950, and AR3 1960 on the right upper ends thereof, andrelative indices suIdxBLtemp on the minimum units BL0 1925, BL1 1935,BL2 1965, and BL3 1975 on the left lower ends thereof may be set astemporary variables.

FIG. 26 illustrates locations of minimum units in the 2N×2N sizedmaximum coding unit 1900 and prediction units 2300 and 2310 each havinga partition type 2N×nU according to an exemplary embodiment.

In the case of the prediction unit 2300 having a partition type 2N×nUand a partition index 0, the Equations (5) and (6) may be used withoutchanges, for indices of a minimum unit AL0 1905 on the left upper end ofthe prediction unit 2300 and a minimum unit AR0 1920 on the right upperend thereof with respect to the maximum coding unit, respectively.suIdxALtemp=ConvertAbsToRel(SuIdxAL);  (5)suIdxARtemp=ConvertAbsToRel(SuIdxUL+CuSize/SuSize−1);  (6)

An Equation (17) may be set based on the Equation (7), for the index ofthe minimum unit BL0 1905 on the left lower end of the prediction unit2300 having the partition type 2N×nU and the partition index 0 withrespect to the maximum coding unit 1900:suIdxBLtemp=ConvertAbsToRel(SuIdxUL+((CuSize/SuSize)>>1−1)*LcuWidthInSus);  (7)suIdxBLtemp−=NumSusInLcu>>3;  (17)

According to the Equations (5), (6), and (17), the temporary variablessuIdxALtemp, suIdxBLtemp, and suIdxARtemp may represent the relativeindices of the minimum unit AL0 1905 on the left upper end of theprediction unit 2300, a minimum unit BL0 1905 on the left lower endthereof, and the minimum unit AR0 1920 on the right upper end thereof,respectively.

In the case of the prediction unit 2310 having a partition type 2N×nUand a partition index 1, Equations (18) and (19) may be set based on theEquations (5) and (6), for the indices of a minimum unit AL1 1925 on theleft upper end of the prediction unit 2310 and a minimum unit AR1 1940on the right upper end thereof:suIdxALtemp=ConvertAbsToRel(SuIdxAL);  (5)suIdxALtemp+=NumSusInLcu>>3;  (18)suIdxARtemp=ConvertAbsToRel(SuIdxUL+CuSize/SuSize−1);  (6)suIdxARtemp+=NumSusInLcu>>3;  (19)

Equations (7) and (8) below may be used for the index of a minimum unitBL1 1965 on the left lower end of the prediction unit 2310 having thepartition type 2N×nU and the partition index 1 with respect to themaximum coding unit:suIdxBLtemp=ConvertAbsToRel(SuIdxUL+((CuSize/SuSize)>>1−1)*LcuWidthInSus);  (7)suIdxBLtemp+=NumSusInLcu>>1;  (8)

According to the Equations (18), (19), and (8), the temporary variablessuIdxALtemp, suIdxARtemp, and suIdxBLtemp may represent the relativeindices of the minimum unit AL1 1925 on the left upper end of theprediction unit 2310, the minimum unit AR1 1940 on the right upper endthereof, and the minimum unit BL1 1965 on the left lower end thereof,respectively.

FIG. 27 illustrates locations of minimum units in the 2N×2N sizedmaximum coding unit 1900 and prediction units 2400 and 2410 each havinga partition type 2N×nD according to an exemplary embodiment.

In the case of the prediction unit 2400 having a partition type 2N×nDand a partition index 0, Equations (5) and (6) may be used for indicesof a minimum unit AL0 1905 on the left upper end of the prediction unit2400 and a minimum unit AR0 1920 on the right upper end thereof withrespect to the maximum coding unit, respectively.suIdxALtemp=ConvertAbsToRel(SuIdxAL);  (5)suIdxARtemp=ConvertAbsToRel(SuIdxUL+CuSize/SuSize−1);  (6)

An Equation (20) may be set based on the Equation (7), for the index ofa minimum unit BL0 1945 on the left lower end of the prediction unit2400 having the partition type 2N×nD and the partition index 0 withrespect to the maximum coding unit 1900:suIdxBLtemp=ConvertAbsToRel(SuIdxUL+((CuSize/SuSize)>>1−1)*LcuWidthInSus);  (7)suIdxBLtemp+=NumSusInLcu>>2+NumSusInLcu>>3;  (20)

According to the Equations (5), (6), and (20), the temporary variablessuIdxALtemp, suIdxBLtemp, and suIdxARtemp may represent the relativeindices of the minimum unit AL0 1905 on the left upper end of theprediction unit 2400, the minimum unit BL0 1945 on the left lower endthereof, and the minimum unit AR0 1920 on the right upper end thereof,respectively.

In the case of the prediction unit 2410 having a partition type 2N×nDand a partition index 1, Equations (21) and (22) may be set based on theEquations (5) and (6), for the indices of a minimum unit AL1 1965 on theleft upper end of the prediction unit 2410 and a minimum unit AR1 1980on the right upper end thereof:suIdxALtemp=ConvertAbsToRel(SuIdxAL);  (5)suIdxALtemp+=NumSusInLcu>>1+NumSusInLcu>>3;  (21)suIdxARtemp=ConvertAbsToRel(SuIdxUL+CuSize/SuSize−1);  (6)suIdxARtemp+=NumSusInLcu>>1+NumSusInLcu>>3;  (22)

Equations (7) and (8) may be used for the index of a minimum unit BL11965 on the left lower end of the prediction unit 2410 having thepartition type 2N×nD and the partition index 1 with respect to themaximum coding unit 1900:suIdxBLtemp=ConvertAbsToRel(SuIdxUL+((CuSize/SuSize)>>1−1)*LcuWidthInSus);  (7)suIdxBLtemp+=NumSusInLcu>>1;  (8)

According to the Equations (21), (22), and (6), the temporary variablessuIdxALtemp, suIdxARtemp, and suIdxBLtemp may represent the relativeindices of the minimum unit AL1 1965 on the left upper end of theprediction unit 2410, the minimum unit AR1 1980 on the right upper endthereof, and the minimum unit BL1 1965 on the left lower end thereof,respectively.

FIG. 28 illustrates locations of minimum units in the 2N×2N sizedmaximum coding unit 1900 and prediction units 2500 and 2510 each havinga partition type nL×2N according to an exemplary embodiment.

In the case of the prediction unit 2500 having a partition type nL×2Nand a partition index 0, the Equations (5), (7), and (8) may be usedwithout changes, for indices of a minimum unit AL0 1905 on the leftupper end of the prediction unit 2500 and a minimum unit BL0 1965 on theleft lower end thereof with respect to the maximum coding unit 1900:suIdxALtemp=ConvertAbsToRel(SuIdxAL);  (5)suIdxBLtemp=ConvertAbsToRel(SuIdxUL+((CuSize/SuSize)>>1−1)*LcuWidthInSus);  (7)suIdxBLtemp+=NumSusInLcu>>1;  (8)

An Equation (23) may be set based on the Equation (6), for the index ofthe minimum unit AR0 1905 on the right upper end of the prediction unit2500 having the partition type nL×2N and the partition index 0 withrespect to the maximum coding unit:suIdxARtemp=ConvertAbsToRel(SuIdxUL+CuSize/SuSize−1);  (6)suIdxARtemp−=NumSusInLcu>>2+NumSusInLcu>>4;  (23)

According to the Equations (5), (8), and (23), the temporary variablessuIdxALtemp, suIdxBLtemp, and suIdxARtemp may represent the relativeindices of the minimum unit AL0 1905 on the left upper end of theprediction unit 2500, the minimum unit BL0 1965 on the left lower endthereof, and the minimum unit AR0 1905 on the right upper end thereof,respectively.

In the case of the prediction unit 2510 having a partition type nL×2Nand a partition index 1, Equations (24) and (25) may be set based on theEquations (4) and (7), respectively, for the indices of a minimum unitAL1 1910 on the left upper end of the prediction unit 2510 and a minimumunit BL1 1970 on the left lower end thereof:suIdxALtemp=ConvertAbsToRel(SuIdxAL);  (4)suIdxALtemp+=NumSusInLcu>>4;  (24)psuIdxBLtemp=ConvertAbsToRel(SuIdxUL+((CuSize/SuSize)>>1−1)*LcuWidthInSus);  (7)suIdxBLtemp+=NumSusInLcu>>1+NumSusInLcu>>4;  (25)

The Equation (6) may be used for the index of the minimum unit AR1 1920on the right upper end of the prediction unit 2510 having the partitiontype nL×2N and the partition index 1 with respect to the maximum codingunit 1900:suIdxARtemp=ConvertAbsToRel(SuIdxUL+CuSize/SuSize−1);  (6)

According to the Equations (24), (25), and (6), the temporary variablessuIdxALtemp, suIdxARtemp, and suIdxBLtemp may represent the relativeindices of the minimum unit AL1 1910 on the left upper end of theprediction unit 2510, the minimum unit BL1 1970 on the left lower endthereof, and the minimum unit AR1 1920 on the right upper end thereof,respectively.

FIG. 29 illustrates locations of minimum units in the 2N×2N sizedmaximum coding unit 1900 and prediction units 2600 and 2610 each havinga partition type nR×2N according to an exemplary embodiment.

In the case of the prediction unit 2600 having a partition type nR×2Nand a partition index 0, the Equations (5), (7), and (8) may be usedwithout changes, for indices of a minimum unit AL0 1905 on the leftupper end of the prediction unit 2600 and a minimum unit BL0 1965 on theleft lower end thereof with respect to the maximum coding unit 1900:suIdxALtemp=ConvertAbsToRel(SuIdxAL);  (5)suIdxBLtemp=ConvertAbsToRel(SuIdxUL+((CuSize/SuSize)>>1−1)*LcuWidthInSus);  (7)suIdxBLtemp+=NumSusInLcu>>1;  (8)

An Equation (26) may be set based on the Equation (6), for the index ofa minimum unit AR0 1915 on the right upper end of the prediction unit2600 having the partition type nR×2N and the partition index 0 withrespect to the maximum coding unit:suIdxARtemp=ConvertAbsToRel(SuIdxUL+CuSize/SuSize−1);  (6)suIdxARtemp−=NumSusInLcu>>4;  (26)

According to the Equations (5), (8), and (26), the temporary variablessuIdxALtemp, suIdxBLtemp, and suIdxARtemp may represent the relativeindices of the minimum unit AL0 1905 on the left upper end of theprediction unit 2600, the minimum unit BL0 1965 on the left lower endthereof, and the minimum unit AR0 1915 on the right upper end thereof,respectively.

In the case of the prediction unit 2610 having a partition type nR×2Nand a partition index 1, Equations (27) and (28) may be set based on theEquations (5) and (7), respectively, for the indices of a minimum unitAL1 1920 on the left upper end of the prediction unit 2610 and a minimumunit BL1 1980 on the left lower end thereof:suIdxALtemp=ConvertAbsToRel(SuIdxAL);  (5)suIdxALtemp+=NumSusInLcu>>2+NumSusInLcu>>4;  (27)suIdxBLtemp=ConvertAbsToRel(SuIdxUL+((CuSize/SuSize)>>1−1)*LcuWidthInSus);  (8)suIdxBLtemp+=NumSusInLcu>>1+NumSusInLcu>>2+NumSusInLcu>>4;  (28)

The Equation (6) may be used for the index of a minimum unit AR1 1920 onthe right upper end of the prediction unit 2610 having the partitiontype nR×2N and the partition index 1 with respect to the maximum codingunit 1900:suIdxARtemp=ConvertAbsToRel(SuIdxUL+CuSize/SuSize−1);  (6)

According to the Equations (27), (28), and (6), the temporary variablessuIdxALtemp, suIdxARtemp, and suIdxBLtemp may represent the relativeindices of the minimum unit AL1 1920 on the left upper end of theprediction unit 2610, the minimum unit BL1 1980 on the left lower endthereof, and the minimum unit AR1 1920 on the right upper end thereof,respectively.

Referring to FIGS. 22 through 29 described above, the relative indicessuIdxALtemp, suIdxARtemp, and suIdxBLtemp of the minimum units on theleft upper end, the right upper end, and the left lower end of a currentprediction unit may be converted into the absolute indices SuIdxAL,SuIdxAR, and SuIdxBL according to Equations (29) through (31):SuIdxAL=ConvertRelToAbs(suIdxALtemp);  (29)SuIdxAR=ConvertRelToAbs(suIdxARtemp);  (30)SuIdxBL=ConvertRelToAbs(suIdxBLtemp);  (31)

Accordingly, referring to FIGS. 22 through 29, the coding unitdeterminer 1420 and the decoder 1530 may determine the indices SuIdxAL,SuIdxAR, and SuIdxBL of the minimum units on the left upper end, theright upper end, and the left lower end of a current prediction unitwith respect to a maximum coding unit by using the index of the minimumunit on the left upper end of prediction units of current coding unit,the size CuSize of a current coding unit, and the partition type andpartition index of the current prediction unit.

FIG. 30 is a flowchart illustrating an image encoding method based on ascanning order of hierarchical data units, according to an exemplaryembodiment.

In operation 2710, a picture is split into coding units each having apredetermined maximum size.

In operation 2720, a maximum coding unit and coding units having ahierarchical structure are determined and a coded depth and a codingmode of the coding units are determined, by encoding the picture basedon the maximum coding unit and the coding units, into which the pictureis split according to depths. A data scanning order of data unitsbelonging to a hierarchical structure in the coding units having a treestructure may be determined based on the absolute location or relativelocation of a lower data unit as compared to an upper data unit. Forexample, the upper data unit may include at least one lower data unit inthe order from a maximum coding unit, which is the uppermost data unit,to a minimum coding unit, which is the lowermost data unit, through adeeper coding unit, a coding unit, a prediction unit, and a partition.

In other words, encoding based on the data scanning order depending onthe absolute locations and relative locations of coding units of atleast one depth having a hierarchical structure may be performed on eachmaximum coding unit in units of regions into which the maximum codingunit is hierarchically split as the depth deepens. Consequently, acoding mode for a coding unit of a coded depth including informationabout at least one coded depth in which a result of the coding on eachregion is to be output may be determined.

In operation 2730, encoding information about a coded depth and a codingmode and size information of the maximum coding unit are output for eachmaximum coding unit. Coded video data may also be output.

FIG. 31 is a flowchart of a video decoding method based on a scanningorder of hierarchical data units, according to an exemplary embodiment.

In operation 2810, a bitstream for an encoded video is received andparsed.

In operation 2820, encoding information about a coded depth and a codingmode of coding units and size information of a maximum coding unit areextracted from a bitstream. Coded video data may also be extracted.

In operation 2830, a hierarchical structure of the maximum coding unitand the coding units is determined and a picture is restored based onthe coding units, by using the encoding information about the codeddepth and the coding mode and the size information of the maximum codingunit. Pieces of image data may be read and decoded in a data scanningorder determined due to the absolute location or relative location of alower data unit determined as compared to the location of an upper dataunit from among data units determined by coding units having a treestructure.

In other words, video data obtained by encoding each coding unit of atleast one coded depth of the maximum coding unit is decoded based on theencoding information about a coded depth and a coding mode for eachmaximum coding unit. The coded video data is read based on the datascanning order depending on the absolute locations and relativelocations of the coding units of at least one depth having ahierarchical structure, and current video data may be decoded byreferring to neighborhood information read and decoded in a scanningorder.

In the video encoding method based on the scanning order of hierarchicaldata units according to the exemplary embodiment of FIG. 23 and thevideo decoding method based on the scanning order of hierarchical dataunits according to the exemplary embodiment of FIG. 24, the absolutelocation of a coding unit and the relative location of the coding unitwith respect to a maximum coding unit may be transformed into each otherand analyzed in the zigzag scanning order of the data units having thehierarchical structure.

In the relationship between an upper coding unit and a lower coding unitin the maximum coding unit, the absolute location or relative locationof the lower coding unit may be determined using the absolute locationor relative location of the upper coding unit. In contrast, the pixelposition or index of a corresponding deeper coding unit or acorresponding prediction unit may be determined based on the absolutelocation of a minimum unit or the index of the minimum unit with respectto the maximum coding unit.

Based on the relative location of a minimum unit located on the leftupper end of an upper data unit, the relative location of a minimum unitat a location different from the left upper end of the upper data unitmay be determined.

The usability of neighborhood information may be determined from thelocation of a current maximum coding unit, the index of a minimum unitof a current prediction unit, and the partition type of the currentprediction unit. For example, it is determined whether neighborhoodinformation of a current prediction unit exists in a current picture orcan be referred to by being accessed in a zigzag scanning order, and theusability of the neighborhood information may be determined, by usingthe x and y coordinate values of a current maximum coding unitdetermined from the location of the current maximum coding unit, the xcoordinate value of a current coding unit determined from the index of aminimum unit on the right upper end of the current maximum coding unit,and the y coordinate value of the current coding unit determined fromthe index of a minimum unit on the left lower end of the current maximumcoding unit.

Accordingly, a video encoding method based on the scanning order ofhierarchical data units according to an exemplary embodiment and a videodecoding method based on the scanning order of hierarchical data unitsaccording to an exemplary embodiment are capable of accurately andquickly analyzing the positions of various types of data units whenusing data units relatively larger than existing macroblocks and thevarious types of data units, by mutually transforming the relativelocations and absolute locations of the data units based on the scanningorder of the hierarchical data units, thereby efficiently encoding ordecoding a video.

Furthermore, one or more exemplary embodiments can be written ascomputer programs and can be implemented in general-use digitalcomputers that execute the programs using a computer readable recordingmedium. Examples of the computer readable recording medium includemagnetic storage media (e.g., ROM, floppy disks, hard disks, etc.) andoptical recording media (e.g., CD-ROMs, or DVDs). Moreover, one or moreunits of the above-described apparatuses can include a processor ormicroprocessor executing a computer program stored in acomputer-readable medium.

While exemplary embodiments have been particularly shown and described,it will be understood by those of ordinary skill in the art that variouschanges in form and details may be made therein without departing fromthe spirit and scope of the present inventive concept as defined by theappended claims. The exemplary embodiments should be considered indescriptive sense only and not for purposes of limitation. Therefore,the scope of the present inventive concept is defined not by thedetailed description of exemplary embodiments, but by the appendedclaims, and all differences within the scope will be construed as beingincluded in the present inventive concept.

What is claimed is:
 1. An encoding apparatus comprising: a maximumcoding unit splitter configured to split a picture into a plurality ofmaximum coding units; a coding unit determiner configured tohierarchically split at least one coding unit of depth including atleast one of a current depth and a lower depth from a maximum codingunit among the plurality of maximum coding units, and when a scan indexof a minimum unit included in a current prediction unit is greater thana scan index of a minimum unit corresponding to a lower-left position,to determine a neighboring unit corresponding to the lower-left positionis available to use for the current prediction unit, wherein the minimumunit corresponding to the lower-left position is located below and tothe left of the current prediction unit in the picture where the currentprediction unit is located; and an outputter configured to generateencoding information regarding the picture based on the at least onecoding unit of the depth, to generate split information which indicateswhether a coding unit of a first depth is split into coding units of asecond depth lower than the first depth, wherein the split informationis generated to be used to hierarchically split the maximum coding unitinto the at least one coding unit of the depth including at least one ofthe current depth and the lower depth, to generate partition informationwhich indicates whether a partition type of asymmetrically splitting thecurrent coding unit along at least one of a height and a width of thecurrent coding unit when the current coding unit is a coding unit whichis not split into coding units, wherein the partition information isgenerated to be used to obtain at least one prediction unit includingthe current prediction unit from the current coding unit to performprediction, and to generate a bitstream including encoding informationregarding the picture, the split information and the partitioninformation, wherein the scan index follows a raster scan order betweenthe plurality of maximum coding units, wherein the scan index follows anupper-left, upper-right, lower-left, and lower-right order betweencoding units included in the maximum coding unit and between minimumunits in a coding unit included in the coding units, and wherein themaximum coding unit has a square shape.
 2. An encoding methodcomprising: splitting a picture into a plurality of maximum codingunits; hierarchically splitting at least one coding unit of depthincluding at least one of a current depth and a lower depth from amaximum coding unit among the plurality of maximum coding units; when ascan index of a minimum unit included in a current prediction unit isgreater than a scan index of a minimum unit corresponding to alower-left position, determining a neighboring unit corresponding to thelower-left position is available to use for the current prediction unit,wherein the minimum unit corresponding to the lower-left position islocated below and to the left of the current prediction unit in thepicture where the current prediction unit is located; generatingencoding information regarding the picture based on the at least onecoding unit of the depth; generating split information which indicateswhether a coding unit of a first depth is split into coding units of asecond depth lower than the first depth, wherein the split informationis generated to be used to hierarchically split the maximum coding unit,among the plurality of maximum coding units, into the at least onecoding unit of the depth including at least one of the current depth andthe lower depth; generating partition information which indicateswhether a partition type of asymmetrically splitting the current codingunit along at least one of a height and a width of the current codingunit when the current coding unit is a coding unit which is not splitinto coding units, wherein the partition information is generated to beused to obtain at least one prediction unit including the currentprediction unit from the current coding unit to perform prediction; andgenerating a bitstream including encoding information regarding thepicture, the split information and the partition information, whereinthe scan index follows a raster scan order between the plurality ofmaximum coding units, wherein the scan index follows an upper-left,upper-right, lower-left, and lower-right order between coding unitsincluded in the maximum coding unit and between minimum units in acoding unit included in the coding units, and wherein the maximum codingunit has a square shape.
 3. An encoding method comprising: splitting apicture into a plurality of maximum coding units; hierarchicallysplitting at least one coding unit of depth including at least one of acurrent depth and a lower depth from a maximum coding unit among theplurality of maximum coding units; when a scan index of a minimum unitincluded in a current prediction unit is greater than a scan index of aminimum unit corresponding to a lower-left position, determining aneighboring unit corresponding to the lower-left position is availableto use for the current prediction unit, wherein the minimum unitcorresponding to the lower-left position is located below and to theleft of the current prediction unit in the picture where the currentprediction unit is located; generating encoding information regardingthe picture based on the at least one coding unit of the depth;generating split information which indicates whether a coding unit of afirst depth is split into coding units of a second depth lower than thefirst depth, wherein the split information is generated to be used tohierarchically split the maximum coding unit, among the plurality ofmaximum coding units, into the at least one coding unit of the depthincluding at least one of the current depth and the lower depth;generating partition information which indicates whether a partitiontype of asymmetrically splitting the current coding unit along at leastone of a height and a width of the current coding unit when the currentcoding unit is a coding unit which is not split into coding units,wherein the partition information is generated to be used to obtain atleast one prediction unit including the current prediction unit from thecurrent coding unit to perform prediction; and generating a bitstreamincluding encoding information regarding the picture, the splitinformation and the partition information, wherein the scan indexfollows a raster scan order between the plurality of maximum codingunits, wherein the scan index follows an upper-left, upper-right,lower-left, and lower-right order between coding units included in themaximum coding unit and between minimum units in a coding unit includedin the coding units, and wherein the at least one coding unit of depthin the maximum coding unit has a square shape.
 4. A non-transitorycomputer-readable storage medium storing a bitstream comprising:encoding information regarding a picture; split information whichindicates whether a coding unit of a first depth is split into codingunits of a second depth lower than the first depth, wherein the splitinformation is generated to be used to hierarchically split a maximumcoding unit, among a plurality of maximum coding units, into at leastone coding unit of depth including at least one of a current depth and alower depth; and partition information which indicates whether apartition type of asymmetrically splitting a current coding unit alongat least one of a height and a width of the current coding unit when thecurrent coding unit is a coding unit which is not split, wherein thepartition information is generated to be used to obtain at least oneprediction unit including a current prediction unit from the currentcoding unit to perform prediction, wherein the encoding information isgenerated based on the at least one coding unit, when a scan index of aminimum unit included in the current prediction unit is greater than ascan index of a minimum unit corresponding to a lower-left position, aneighboring unit corresponding to the lower-left position is determinedto be available to use for the current prediction unit, wherein theminimum unit corresponding to the lower-left position is located belowand to the left of the current prediction unit in the picture where thecurrent prediction unit is located, wherein the scan index follows araster scan order between the plurality of maximum coding units, whereinthe scan index follows an upper-left, upper-right, lower-left, andlower-right order between coding units included in the maximum codingunit and between minimum units in a coding unit included in the codingunits, and wherein the maximum coding unit has a square shape.
 5. Anon-transitory computer-readable storage medium storing a bitstreamcomprising: encoding information regarding a picture; split informationwhich indicates whether a coding unit of a first depth is split intocoding units of a second depth lower than the first depth, wherein thesplit information is generated to be used to hierarchically split amaximum coding unit, among a plurality of maximum coding units, into atleast one coding unit of depth including at least one of a current depthand a lower depth; and partition information which indicates whether apartition type of asymmetrically splitting a current coding unit alongat least one of a height and a width of the current coding unit when thecurrent coding unit is a coding unit which is not split, wherein thepartition information is generated to be used to obtain at least oneprediction unit including a current prediction unit from the currentcoding unit to perform prediction, wherein the encoding information isgenerated based on the at least one coding unit, when a scan index of aminimum unit included in the current prediction unit is greater than ascan index of a minimum unit corresponding to a lower-left position, aneighboring unit corresponding to the lower-left position is determinedto be available to use for the current prediction unit, wherein theminimum unit corresponding to the lower-left position is located belowand to the left of the current prediction unit in the picture where thecurrent prediction unit is located, wherein the scan index follows araster scan order between the plurality of maximum coding units, whereinthe scan index follows an upper-left, upper-right, lower-left, andlower-right order between coding units included in the maximum codingunit and between minimum units in a coding unit included in the codingunits, and wherein the at least one coding unit of depth in the maximumcoding unit has a square shape.
 6. A non-transitory computer-readablestorage medium storing a bitstream comprising: encoding informationregarding a picture; split information which indicates whether a codingunit of a first depth is split into coding units of a second depth lowerthan the first depth, wherein the split information is generated to beused to hierarchically split a maximum coding unit, among a plurality ofmaximum coding units, into at least one coding unit of depth includingat least one of a current depth and a lower depth; and partitioninformation which indicates whether a partition type of asymmetricallysplitting a current coding unit along at least one of a height and awidth of the current coding unit when the current coding unit is acoding unit which is not split, wherein the partition information isgenerated to be used to obtain at least one prediction unit including acurrent prediction unit from the current coding unit to performprediction, wherein the encoding information is generated based on theat least one coding unit, when a scan index of a minimum unit includedin the current prediction unit is greater than a scan index of a minimumunit corresponding to a lower-left position, a neighboring unitcorresponding to the lower-left position is determined to be availableto use for the current prediction unit, wherein the minimum unitcorresponding to the lower-left position is located below and to theleft of the current prediction unit in the picture where the currentprediction unit is located, wherein the scan index follows a raster scanorder between the plurality of maximum coding units, wherein the scanindex follows an upper-left, upper-right, lower-left, and lower-rightorder between coding units included in the maximum coding unit andbetween minimum units in a coding unit included in the coding units,wherein the maximum coding unit has a square shape, and wherein the atleast one coding unit of depth in the maximum coding unit has a squareshape.
 7. A non-transitory computer-readable storage medium storing abitstream comprising: encoding information regarding a picture; splitinformation which indicates whether a coding unit of a first depth issplit into coding units of a second depth lower than the first depth,wherein the split information is generated to be used to hierarchicallysplit a maximum coding unit, among a plurality of maximum coding units,into at least one coding unit of depth including at least one of acurrent depth and a lower depth; and partition information whichindicates whether a partition type of asymmetrically splitting a currentcoding unit along at least one of a height and a width of the currentcoding unit when the current coding unit is a coding unit which is notsplit, wherein the partition information is generated to be used toobtain at least one prediction unit including a current prediction unitfrom the current coding unit into to perform prediction, wherein theencoding information is generated based on the at least one coding unit,when a scan index of a minimum unit included in the current predictionunit is greater than a scan index of a minimum unit corresponding to alower-left position, a neighboring unit corresponding to the lower-leftposition is determined to be available to use for the current predictionunit, wherein the minimum unit corresponding to the lower-left positionis located below and to the left of the current prediction unit in thepicture where the current prediction unit is located, wherein the scanindex follows a raster scan order between the plurality of maximumcoding units, wherein the scan index follows an upper-left, upper-right,lower-left, and lower-right order between coding units included in themaximum coding unit and between minimum units in a coding unit includedin the coding units, and wherein the scan index is increased with theraster scan order between the plurality of maximum coding units.